Interest and/or penalties paid to the IRS are not deductible on your tax return.
Per IRS Publication 529 Miscellaneous Deductions, on page 7:
Fines or Penalties
No deduction is allowed for fines and penalties paid to a government or specified nongovernmental entity for the violation of any law except in the following situations.
Nondeductible amounts include an amount paid in settlement of your actual or potential liability for a fine or penalty (civil or criminal). Fines or penalties include amounts paid such as parking tickets, tax penalties, and penalties deducted from teachers' paychecks after an illegal strike.
Per IRS Notice 746 Penalty and Interest, on page 4:
Beginning with tax year 1991, you can no longer deduct interest you paid to the United States Treasury as an itemized deduction on your Form 1040, Schedule A.
Note that any link in the information above is updated each year automatically and will take you to the most recent version of the document at the time it is accessed.