Just print out your tax returns and mail them. It's what I've done for 90% of the tax returns in my lifetime, and I always did it for state because there was always an extra free for e-filing of state taxes.
As long as you mail the taxes on April 17th, with a check, you will be okay. The taxes just have to show a postage mark of April 17th or earlier.
If you have to go to the bank and get a certified check and mail the taxes insured / certified mail, that would be one way to do it. The bank should have given you some temporary checks when you opened the account - that could work, too.
