...can I still count it as a personal day as long as I am not doing work on the property itself?
Yes, as long as you are not engaged in repair and maintenance, on a substantially full time basis, on the property, the day would count as a personal use day when you are using it for personal purposes (including working at your primary job).
