No. You will just have less income to pay tax on when you take it out. Unless it goes back up, of course.
