The gain/loss calculation must always be done for a return of contribution before the due date of the tax return. The fact that an amount equal to the money deposited as 2023 contributions remained uninvested is irrelevant. What is relevant is the overall investment performance in the HSA. Once deposited into the HSA, the 2023 contributions are no longer independent of the other funds in the account. If any part of the HSA is invested, there will be a nonzero gain or loss attributable to the 2023 contributions.
Losing money in the investments doesn't help. To be a valid return of contribution the gain/loss calculation must be done. If more than the loss-adjusted amount is distributed, the amount in excess of the loss-adjusted amount constitutes a regular distribution that you can either apply to qualified medical expenses or you can treat as taxable and subject to penalty.
I've already described what must be done. There is no easier way (unless you are willing to pay several thousand dollars in otherwise unnecessary taxes and penalties).
