I'm not sure if I need to include my 1098-E form for student loan interest when filing my 1040 paper return.
No, you do not need to attach form 1098-E to you tax return when you file by mail.
You only attach copies B of your forms W-2.