Can my claimed tax dependent children use of non-custodial parents HSA funds without being claimed on their taxes?

I have been divorced 9 years and have primary custodial custody and financially support/provide 50% or more of the needs/requirements of the children (primary custody 1000 miles away from the other parent for no less than 75% of the year - 255 days). I have therefore claimed the kids as my dependents on taxes every year for the child tax credit and refunds. Also, my ex makes approximately 75% more annually and the child tax credit was no benefit of those returns, so it was agreed my refund would better support the children each year if I claimed and received a refund.

 

There is an Employer Contributed HSA account and insurance coverage. The children remained on the non-custodial parents' insurance policy and continued using the HSA account for Approved Medical Expenses, per our divorce agreement. I also qualified for and used state Medicaid for the children to limit the use of the HSA account. Our agreement states the HSA account will pay all approved medical expenses 100% until depleted and then expenses are to be split per child support calculations on the dollar. I pay OOP for most things in order to get credit card points, then request a reimbursement for the approved expenses from the HSA at the beginning of the following year after I prepared my taxes and properly sorted HSA paid and personal paid for deductions. I also supply support documents to show approved HSA expenses. We have done this for 15 plus years, even when married, and never had issues. My ex is now claiming that the HSA was illegally used all these years, and the children need to be claimed as dependents by them in order to use the HSA funds and is demanding I personally re-pay all funds used plus the 20% penalties/fees, as well as amend my taxes for the last 3 years and also pay all those penalties and fees. 

 

I need clarification on

A) I am legally in the right to have claimed, and continue claiming, the children based on being the primary custodial parent and provide 50% minimum financial support, or

B) does the rule of "only a claimed dependent" is legally able to use the HSA funds and in fact could VOID my status A, meaning I could potentially have to be out thousands of dollars to re-pay and amend all my taxes?

C) One child is 22 and files their own taxes, so would that child have to also amend their taxes in order to be "compliant" with the HSA rule? They are not a FT student.

 

Any and all clarification, IRS Publication references are welcome and/or specific codes, rules, laws, etc. that I can be directed to are appreciated!