'In 2022 we bought new windows for the house. We didn't make enough to have to file taxes, however should we file in order to get the tax deduction on the windows?

The credit can reduce your tax liability, but if you owe no tax (such as because your income is low), then you won’t benefit from filing.  

The Energy Efficient Home Improvement Credit can only be used to reduce your tax liability on your federal tax return.  If you do not have any tax liability the credit cannot be used.  The credit is only available in the tax year when the energy efficient windows were installed.