Tax filing deadline

I always thought April 15 is the deadline for filing tax returns and had to scramble to meet the deadline to avoid any penalties. However, I realized that is not true for those with tax refund. When you will be getting tax refund, you don't have to file by April 15(although it is good to file on time). You actually have three years to file the return when you expect refund from the government. April 15 deadline only applies for those who owe taxes. For those expecting refund, you don't have to worry about the April 15 deadline as long as you file within three years.Is my understanding correct?