Paper return still not processed

I filed for paper return and I understand that this will take longer than electronic returns. However, it has been 4 months and on IRS it still shows that it is not processed. I don't have any refund and in fact owe IRS money and I have paid through direct deposit. Should I be worried?