If you owe unemployment compensation to the state because the state believes that you were previously overpaid, then, yes, the IRS can take part or all of your refund if the state asks them to.
See this article on MSN about this issue - https://www.msn.com/en-us/money/taxes/6-ways-the-irs-can-seize-your-tax-refund/ar-BBAY5DY
Note that the IRS normally will send you a letter telling you that they have done this, but the letter might not arrive until long after the refund should have come.
Your only other alternative is to contact the IRS. See https://www.irs.com/articles/contact-the-irs
