The mortgage payoff is not a part of the capital gain or capital loss calculation on a personal residence.
Gain or Loss = Sales Price minus Sales Expenses minus Adjusted Basis (Purchase Price plus the cost of improvements prior to the sale)
