Interest and/or penalties paid to the IRS are not deductible on your tax return.

Per IRS Publication 529 Miscellaneous Deductions, page 15:

Fines or Penalties
You can't deduct fines or penalties you pay to a governmental unit for violating a law. This includes an amount paid in settlement of your actual or potential liability for a fine or penalty (civil or criminal). Fines or penalties include parking tickets, tax penalties, and penalties deducted from teachers' paychecks after an illegal strike.

Per IRS Notice 746 Penalty and Interest, page 4:

Interest Paid
Beginning with tax year 1991, you can no longer deduct interest you paid to the United States Treasury as an itemized deduction on your Form 1040, Schedule A.