You can deduct expenses for travel from your home and a temporary work site outside the metropolitan area where you live. Temporary means you are not expected to be there more than a year. Also, if the work site is your regular work site (more than a year) but it is remote and your family would not be able to reasonably live there, then the travel is deductible.
The down time at the job location would not be deducted from the days counted for per diem meals. You can use the per diem meal allowance but not the lodging per diem. The IRS does not specifically mention that you "can't" use the meal allowance if a free meal is offered but not accepted. Here is a link to the GSA website Per
diem rates lookup. Click this link to read more: Per diem FAQs
