The property in question was originally purchased as our primary home in 2009 and refinanced 3 times, most recently in 2017 when it was converted to a second home.
Converting a primary residence to a 2nd home has no tax implications. It doesn't change the fact that the only things you can claim on a 2nd home are mortgage interest and property taxes as a SCH A itemized deduction. But with the 2018 TCJA that went into effect on Jan 1 2018, you are limited on the amount of SALT (State and Local Taxes) that you can claim on the SCH A.
We took cash out on the last round for a down payment on our new primary home. Per recent rule changes, my understanding was/is that the cash out portion is treated as a personal loan since the new main home we bought with it is not secured by the debt.
Actually, it "is" secured by the debt. However, you did not use the cash out money on the same property that secures the debt. That's why you are unable to claim the interest deduction on the cash out amount.
For that reason, only about 2/3 of the mortgage interest is deductible now and the discount points we paid at closing were amortized over the life of the loan and also scaled accordingly.
Points paid for your primary residence are fully deductible in the tax year you pay them. They are not amortized over the life of the loan. But any points paid on the "cash out" amount are not deductible at all.
Last year, we converted the property to rental use (put in service on January 1, 2019) and no longer have any personal use (rented all year).
I assume you are referring to the original property you originally purchased in 2009.
For the sake of argument, the adjusted basis of the property is approximately $250K, the loan balance prior to the most recent refinancing was approximately $160K, and roughly $80K was cashed out.
Your cost basis on the property is the lesser about of a) what you paid for the property when you originally purchased it in 2009, or b) the fair market value of the property on the date it was placed in service as a rental. I seriously doubt the value of your property when you placed it in service in 2019 was less than what you originally paid for it in 2009. So your cost basis for depreciation is what you originally paid for the property when you purchased it in 2009.
For personal use properties (i.e. main home or second home) the limitations for these deductions were handled on the worksheets with schedule A, but I'm failing to see any place to limit these values (at least automatically) for schedule E.
There is no SALT limits for business use property - which is precisely what rental property is.
Do I need to manually calculate the reduced interest amount and use that instead of the full amount reported on form 1098?
No. The SALT limits do not apply to rental property.
For the points, would I just reduce the initial cost ammortized proportionally?
No. The SALT limits do not apply to rental property.
But this does not change the fact that your cost basis for the purpose of depreciation, is what you originally paid for the property when you purchased it in 2009.
If you refinanced this property in years past (including before you converted it to a rental) and the cash out amount was not used to improve, repair or maintain this rental property, then the interest paid on the cash out amount is not deductible as a rental expense. If it was used to improve any other "personal use" real estate that you own (primary residence, 2nd home) then it would be a SCH A itemized deduction subject to the SALT limits.
