If I filed a Form 8606, but then did not end up contributing, do I need to file an amendment? And if so, does it need to be completed by 4/15/25?

You need to amend,

in Explanation explain why 8606 is dropping out of your tax return.

It should be done before you file your 2025 tax return not by April 15th.

After Oct 15, e-File of 1040-X stops.