You have to add the closing costs to your cost basis.  This reduces your gain (or increases your loss) by the amount of your closing costs.

Your basis is essentially your investment in an asset—the amount you will use to determine your profit or loss when you sell it. The higher your basis, the less gain there is to be taxed—and therefore, the lower your tax bill. This is why it's so important to accurately track the basis of any investment you own.

Your adjusted basis is generally your cost in acquiring your home plus closing costs at purchase and sale, the cost of any capital improvements you made, less casualty loss amounts and other decreases.