Yes, otherwise the IRS may send you a bill for the tax you would owe if you kept the money. The IRS won't necessarily know it was a rollover to a new qualified plan, unless you tell them.
