I just tried this with the 2021 desktop version. There was no delete button at the bottom, but I was able to delete the form using the Cmd-Delete key combination.
