Help me understand Form 8829 Asset Entry Worksheet -cost must be entered-what cost?

The cost of the house.  If you are taking home office expenses, depreciation on the house is part of those expenses.  To calculate the depreciation, you have to enter the cost basis (original purchase price plus improvements over the years.)