No, you won't be audited because of differing addresses on your driver's license and filing address.
The IRS cares about the income, dependents, deductions, credits and foreign assets you report on your return (or don't report), not whether you have a different mailing address than what is on your driver's license.