Do I need to allocate the sales proceeds and expenses to all the improvements that I have capitalized (Assets) on a rental property besides the main asset and land?

Yes, if the improvements went along with the sale. The easiest way is to allocate the remaining values to the individual assets and then allocate the remaining amount between the land and the building unless this would be unrealistic for the land and building values. The other option is to allocate the sales proceeds to the corresponding asset. For example, if the cost to replace the roof was 5% of the building's value at the time, allocate 5% of the sale to the roof.