Yes, he would be.
Assuming the trust is now irrevocable and you do not have the requisite authority or power, the trust is treated as a separate entity for federal income tax purposes (i.e., not a grantor trust).
You should check with a local attorney who can review the document.