No. You will end up with a higher tax bill. You are required by federal law to depreciate rental property. If you do not, then in the tax year you sell the property you are required to reduce your costs basis in the property by the amount of depreciation you "should" have taken, thus increasing your taxable gain on the sale. That's your penalty for not depreciating the property as required.