Your starting basis is the stepped up value of the house on the date of your parent’s death. That’s your starting basis, because you and your sibling each started with half the FMV as the basis for each half of the ownership interest. Even if your sibling gave you their head to the house, they would’ve “given” you their half of the basis as well.
If you want an even higher basis, that is going to be very difficult to prove, and I don’t think you can prove it from your facts. Consider the example where the home was (for some reason) inherited by two completely unrelated people. If you bought out the other owner for more than their basis (the FMV on the date of death) that would increase your basis. If you had bought out your sibling for more than your siblings basis, that could increase your basis, but you have to prove that it was a fair transaction and you paid the same price as you would have paid in an arm’s length transaction to an unrelated person.
But you apparently bought out your sibling for equal or less than the FMV, and you don’t have the right paperwork. You don’t have to decrease the basis of the home, but you can’t increase it further. Your basis is the FMV on the date of your mother‘s death.
