Yes, when there is an offset, it usually takes longer for the IRS to process your refund because there is an extra step involved.
Fortunately, most of the process is automated so in my experience it normally doesn't take too much longer than a return without an offset.
