Did I understand that correctly?
I don't think you did understand that correctly. Your property improvements are not deductible per-se. They add to the total cost basis of your property.
Lets say you originally paid $100,000 for your property when you purchased it. A few years later you decide to split that 4 bedroom 2 bath house into 2 units. That means you have to provide a separate kitchen so that each unit has it's own cooking, bathing, and other facilities that make it a completely separate living unit. Let's say the cost of do that costs you $20,000. Now, your cost basis in the property is $120,000.
Assuming you will live in one unit as your primary residence and rent out the other unit, and further assuming that each unit occupies exactly 50% of the total floor space, your cost basis in the rental unit is $60,000. After alocating a portion of that to the land (which is never depreciated) the remaining amount will be depreciated over the next 27.5 years.
Understand that depreciation is not a permanent deduction. Whenever you sell or otherwise dispose of the entire property in the future, that depreciation has to be recaptured in the tax year you sell the property. That recaptured depreciation is added to your AGI in the tax year of the sale, and has the potential to bump you into th next higher tax bracket too. Also, you will pay taxes on that recaptured depreciation in the tax year of the sale.
This is why I always do my best to keep the depreciation I"m required to take by law, as low as possible. While it may help my taxes now, there's the potential for it to really bite my arse in the future when I sell the property.
