When did you buy the home? Because the VA funding fee is a special category of mortgage insurance that is fully deductible in the year you close. If it was 2013 or later, you could amend your prior tax returns to deduct the entire amount in year one and then remove the deduction in subsequent years. (It is too late to amend a 2012 or earlier tax return to get an additional refund.)
Unfortunately, if you are amortizing the premium deduction and you pay off a loan early, such as by refinancing, you can't deduct the remaining balance.
