Withdraw the deposit immediately, and make sure the withdrawal includes any earnings on that deposit too. Then at tax filing time you'll be able to indentify it as the withdrawal of an over contribution, and will only be taxed on the earnings, since you're already taxed on the base amount deposited.
