Yes, that is correct. Until the property is available for rent you do not have any deduction on your tax return.
Capital improvements become part of the cost basis of the investment property and the combined total will be the depreciable rental asset. The land value must be separated out because it will always be an appreciable asset under the tax law. For your reference you can use the property tax assessment records to determine the percentage of the cost that should be allocated to land. TurboTax will ask you the land value when you begin your rental activity in 2021.
Utilities, Insurance, etc: Costs you incur before you are actually in business of a rental activity are called start-up expenses. Special tax rules govern the deduction of these costs. Any expense that would be deductible as an operating expense by an ongoing business is a start-up expense when it’s incurred before a business begins.
Start-up Expenses: You can deduct up to $5,000 in these expenses the first year your rental is available for rent. For the past several years this limit has been $5,000. You’ll have to deduct any additional start-up expenses in excess of the first year limit in equal amounts over the first 180 months (15 years) you’re in business. This is referred to as an amortization deduction and is similar to depreciation but somewhat different. TurboTax will help you with this deduction.