No, you won't need to pay it back. First, you are not responsible for the gift tax in that situation. Second, there is no such a provision that might require you to pay back the excluded gift taxes.
