Yes, they can take it all, unless your spouse filed injured spouse relief.
The following is from the Economic Impact Payment Information Center on the IRS web site.
Q20. If I owe tax, or have a Payment agreement with the IRS, or owe other federal or state debts, will my Payment be reduced as an offset?
A20. No, with one exception. The Payment will be offset only by past-due child support. The Bureau of the Fiscal Service will send you a notice if an offset occurs.
If you are married filing jointly and you filed an injured spouse claim with your 2019 tax return (or 2018 tax return if you haven’t filed your 2019 tax return), half of the total Payment will be sent to each spouse and your spouse’s Payment will be offset only for past-due child support. There is no need to file another injured spouse claim for the Payment.