Let's start with what was possible, then you can decide if it is worth fixing.
Let me also say that when you place a property in service, everything in that property is depreciated over 27.5 years, as part of the adjusted cost basis of the property. But if you replace items, they get listed as separate assets, and may have different depreciation schedules depending on their expected life.
So let's run an example. You buy a duplex in 2020 and start renting unit 1 (unit 2 is your personal residence). You should list unit 1 for depreciation with half your cost basis. In 2021, you buy a hot water heater that serves both units. For unit 1, you can take 50% of the cost as a safe harbor deduction, or you can list 50% of the cost of the hot water heater for depreciation. But you probably should use 10 years, not 27.5 years, because that's the normal useful life. The other 50% of the cost is added to the cost basis of unit 2, your personal home.
Then in 2024, you convert unit 2 to a rental. Your basis for depreciation is either your adjusted cost basis, or the fair market value, whichever is less. And that adjusted cost basis includes 50% of the water heater, since it was a property improvement. You can't take the safe harbor for it. You can only depreciate it as included in the overall cost basis of the unit.
Let's carry this forward to 2034, 10 years from now, and assume you don't change accounting methods. The hot water heater fails and is replaced. Because 50% of the hot water heater is a separate asset for unit 1, you can roll up the remaining depreciation and deduct it in 2034, which is what happens when you dispose of a worthless asset that still has depreciation left. The 50% of the water heater that is included in the basis of unit 2 doesn't get any special treatment, you continue to depreciate unit 2 over 27.5 years from its original basis. However, you then have the choice of expensing or depreciating the new water heater in 2034, depending on what the cost is (and whether the safe harbor limits get an inflation adjustment).
Since half the water heater must be depreciated over 27.5 years as part of the overall cost basis for unit 2, the most you can get back is half the cost that could have been a safe harbor deduction in 2021 (minus the depreciation you claimed that now gets reversed). And even if you leave everything alone, you don't have to wait 24 years to get the cost of the water heater back, since you will get a roll-up of leftover depreciation when the water heater is next replaced (which will probably be sooner than 24 years.). So even if the water heater cost the full $5000 and was eligible for a full $2500 safe harbor, the most you will get back is a $2200 expense deduction, which would save you $450 of taxes, against the cost of preparing and filing the form.
