I can remove penalty for excess contributions to an IRA before April 15. How do I do this without being charged early withdrawal penalty by the bank?

Any early withdrawal penalty imposed by the bank is between you and the bank.  It has nothing to do with your tax return.  Unless you IRA agreement with the bank allows an excess contribution to be removed without incurring an early withdrawal penalty, you are stuck with the early withdrawal penalty if you obtain the distribution.