The mortgage payoff is irrelevant ... it is not entered anywhere as it is not part of the calculation. What you actually netted on the sale is also irrelevant.
Basic formula ...
Purchase price + cost to purchase + improvements + cost of sale = basis
Sale price - basis = cap gain or loss.
