You are correct. The answer would be yes that you used it for something else.
Since the land was not part of the home, it would not be part of the buy, build, or substantially improve your home. It would actually be a separate lot from the home that secures the loan if you were able to purchase it separately.