Rental Property Loss and twice as much due to the IRS

So 2023 was our 1st year as landlords. We rented out a condo that had previously been a 2nd home.  We make more that $150,000 combined income.  We both have full time jobs. Our passive income on the condo was $20,000. Our passive losses, mortgage, HOA, property tax, management fees, insurances we're over $26,000, plus depreciation.  So not only do we have over a $6000 net loss (not including our depreciation)! our IRS bill is almost $3500 more than last year. I think mostly in part because we are unable to deduct the mortgage interest from our W2 income.  Does this seem right?  Shouldn't we be able to carry over the amount of mortgage interest we weren't able to deduct from the passive income on Schedule E to our non-passive income on Schedule A?  I mean, it only seems right, and fair. Of course we're old enough to know that life (and the IRS) isn't always fair.  We're only a couple years from retirement. We thought having a rental would help us keep more of our income to put toward retirement.  This is really hurts.  Is there anything we can do or is this how it is?