That's hard to say. I can see them doing it either way, though I think they will probably send it back to you with the unsigned return. But sometimes unsigned returns wind up being accepted, slipping by the IRS.
If they do cash your check and send your return back, they'll put the payment under your Social Security number so you'll get credit for it (in effect, it would be an amount waiting for a tax return). If they send it back to you, then you'll mail it again with your signed return.
Either way, you'll receive credit for the payment.