That's complicated. As a practical matter, you only need to pay if you plan to return to the US where they could sue you. If you plan to remain overseas then there's not much they can do, is there?
Then, what tax are they asking for? Presumably they are asking for payroll taxes, specifically social security (FICA) and Medicare tax. Foreign students in the US for less than 5 years are exempt from social security and medicare tax as long as the work you performed was related to the reason for your Visa.
https://www.irs.gov/individuals/international-taxpayers/foreign-student-liability-for-social-securit...
For example, if you were paid a stipend by your graduate school, or worked at an internship for college credit, that is exempt from social security and medicare. If you had a part time job on the side working for a retail store or something like that (which was probably not permitted by your visa conditions) then that job would not be exempt from social security and medicare tax.
You would also owe social security and medicare tax starting on your 6th year in the US, even if you were still on a J-1 visa, and working a job related to the visa, in most cases. The employee share of these taxes are 6.2% for social security and 1.45% for Medicare.
You would have owed federal income tax and probably state income tax, but if the employer failed to withhold that tax, you already covered it when you filed your tax return -- your tax return calculates the tax you owe, and if you did not have enough withholding, you pay extra when you file your tax return. There is no purpose or legitimacy to them asking for federal or state tax payments (but I think they are probably asking for social security and medicare tax).
