Yes, you can do that if you remain unmarried. Each of you can claim one dependent and can claim up to $5000 of FSA benefits toward that dependent's care.
For the current year, note that you can't claim any benefit unless you claim the child as a dependent. If you have been each paying $2500 toward care of the same dependent, one of you is going to lose that money unless you pay for care for the new child.
(If you each pay $2500 for the care of 3 year old "Adam", only the parent who claims Adam as a dependent can actually qualify. The parent who paid for care of Adam and does not claim Adam as a dependent will find that the $2500 is added back to their taxable income because it is an unqualified use of the account. However, if the other parent claims newborn "Betty" as a dependent, that parent can also claim reimbursement for expenses for the care of Betty.)
