All your “income” is presumed taxable, unless you prove otherwise. In the unlikely event that you are subjected to a full compliance audit, the IRS can ask you to prove the source of every bank deposit and show that it is either taxable income, or that it is some type of nontaxable gift or other non-taxable transfer. However, full compliance audits are extremely rare.
There is also a banking regulation that requires the banks to report to the IRS any money transfers of $10,000 or more. This is mainly to help prevent money laundering, and the IRS will normally not assume that such transfers are taxable, and they rely on taxpayers being honest with their tax returns. However, if you need to transfer more than $10,000 and you do it in several smaller steps to avoid the reporting rules, that can actually constitute a separate crime called “structuring“ even if the underlying purpose of the transfer is totally legal.
As far as mortgage lending is concerned, that does not involve tax regulations and so this forum will be less able to give solid information. In my own past experience, the banks that I have applied to for mortgages have wanted to see the source of any large or unusual deposits that were made to my account in the two or three months prior to my mortgage application. This is not a tax problem, but the mortgage lender wants to be sure that I have not taken out an undisclosed loan that would make it harder for me to re-pay my mortgage. My mortgage lenders have asked me to provide proof that large deposits were either a withdrawal from my retirement account, or a gift from my parents, or the proceeds of some other large sale, so that they can be confident that I do not have undisclosed obligations that would make it harder for me to repay the mortgage.
