Well, first "we" can't have an IRA or 403(b). Each account is owned by a single individual. So if you are married and spouse A owns an IRA and spouse B owns a 403(b), they can't be combined. Spouse B could rollover their 403(b) to their own IRA, but they can't be combined.
Maybe your language was just sloppy, but just to make sure you know.
Then, a traditional IRA is composed of pre-tax contributions and the basis is always zero. Gains don't add to the basis and losses don't subtract from the basis. Nothing that happens in a traditional IRA means anything until you take money out and then it is taxed as ordinary income. The only reason to track market value is to track the performance of your investments, it has no effect on your taxes.
You only have a basis in a traditional IRA if you made non-deductible contributions. These would be tracked on form 8606. If you made non-deductible contributions you need to save copies of your tax returns including your form 8606s indefinitely, so you can prove your basis when you start taking money out.
https://www.irs.gov/forms-pubs/about-form-8606
Your most recent form 8606 would list the non-deductible contribution for that year as well as the total of previous non-deductible contributions, so when you withdraw from the IRA, you really only need that most recent form 8606. If you did not save it, you can get a copy from the IRS.
https://www.irs.gov/individuals/get-transcript
Basically, traditional IRAs are always pre-tax so you pay tax on all your withdrawals. If you had some non-deductible contributions, then a portion of your withdrawals would be tax-exempt (since they were already taxed). But you have to prove it, regardless of whether the IRA custodian kept records. And you still don't need to keep track of market value, just the amount of deductible and non-deductible contributions, which you should have been doing on your tax returns.
----------
Likewise, a 403(b) account is either 100% pre-tax (no basis) or 100% after tax (if you had a Roth option 403(b). Roth option is recent and if you have had the account a long time, it's probably a regular pre-tax 403(b).). There is no legal way to mix pre-tax and after-tax funds in a single 403(b) account.
(I currently have four 403(b) accounts. Pre-tax from employer A (former), pre-tax from employer B (pre-tax standard retirement plan), Roth from employer B (my after-tax contributions), and pre-tax from employer B (a small incentive match account). They are all with the same trustee but each account only contains 100% pre-tax or 100% after tax money.)
Because a 403(b) contains only pre-tax money, all withdrawals are subject to regular income tax, you have no basis. Any gains or losses don't matter for tax purposes, the only thing that matters is how much you withdraw. If you have gains, you withdraw more and pay more tax, if you have losses, you withdraw less so you pay less tax.
----------
If you have a pre-tax IRA with zero basis and a pre-tax 403(b) with zero basis and they are owned by the same person, there is no problem rolling over the 403(b) into the IRA. It's likely that the IRA has higher maintenance fees but a broader range of available investment choices, so that's something to consider.
If you have a traditional IRA with a partial basis due to non-deductible contributions, you could still roll the 403(b) into an IRA account but I would put it into a new IRA so you aren't mingling pre-tax and after-tax funds any more than necessary.
----------
If you have a 403(b) and a separate IRA, when you reach RMD age (72) you will have to make at least a minimum withdrawal from each account. If you roll over the 403(b) into an IRA (the same or a different IRA account) then you can aggregate your IRA balances when calculating your RMD.
