Thanks for the suggestion.
Since the homestead was not technically used as part of the business and wasn't used as a rental (although closest option for type of asset was 'residential rental real estate' and it had never been depreciated. Thus had 0% for 'Percentage of business use' on the "Federal Asset Entry Worksheet". When changed that to 100% the basis was subtracted correctly from the sales price.
