I pressed pay with refund and it is still trying to make me pay with my card

TT wants to get paid. They use your credit card for backup in case something goes wrong with your refund and they can't deduct your payment from it.

It depends.  Here are some possible reasons.

1. Insufficient Refund Amount: If your federal refund isn't enough to cover the TurboTax fees and the additional processing fee for using "Pay with Refund," the system may default to asking for a credit card. TurboTax does charge $40 for you to pay with your refund in addition to the normal fee for using the product. 
2. Filing Type: If you're filing a paper return, using a foreign address, or filing only a state return after already filing your federal return, the "Pay with Refund" option might not be available.

You might consider paying with your card to avoid the $40 fee for having the TurboTax fee deducted from your refund.