since it wasn't our "primary residence," but a "2nd home" and it wasn't used as a rental for income, do we need to pay a tax on the $50,000 gain?
Yes.
IF a required amount of time staying with my in-laws IS required to eliminate the federal tax,
Unfortunately, that's irrelevant. At no time during your period of ownership was the property your primary residence.
