I'm not clear on the facts.
From 2018 to 2021, no depreciation should have been claimed because it was only your personal residence. From 2021 through today, your brother could depreciate a percentage of the duplex equal to the percentage being rented out (on a square foot basis), but 100% depreciation was never allowed as long as he was using one unit as his personal home. You and your brother need to see an accountant to help you file a form 3115 to correct the erroneous depreciation.
(Or, if you and your brother lived together in unit #1 and unit #2 was always rented, then you could have depreciated only unit #2, on a percentage basis. You still need an accountant to help you file form 3115 to correct the error. As I said, the facts are not entirely clear.)
Also, you never depreciate 100% because land doesn't depreciate. You have to subtract the value of the land. Suppose that whenever it became a rental, the cost basis (what you paid in 2018 plus any permanent improvements) was $200,000, and the land was valued at $40,000. Your basis for depreciation would be $160,000, and if the units were equal sized, your brother could list the rented half the property ($80,000) for depreciation, but only when it was actually rented or available for rent.
As far as taxes when the home is sold are concerned, you will always pay recapture first. It works like this:
Suppose you sell in 2025 for $400,000. Your original cost was $200,000, and the correct depreciation ($80,000 starting value over 27.5 years is $2909 per year, 4 years would be $11,636. (If unit #2 was rented since 2018, then you can recalculate my example using 7 years of depreciation, but the basic calculation is the same.)
Your basis is reduced by depreciation, so your basis is now $188,363 and your gain is $211,636. If you own the property equally, then each sibling's basis is $94,181 and each sibling's gain is $105,818.
The first $5,818 of gain is taxable to each brother as depreciation recapture. For you, the remaining $100,000 of gain is fully taxable because the home was not your personal residence for 2 of the previous 5 years. As long as your brother is still living in one unit as a personal residence, his $100,000 of gain is covered by the personal exclusion.
