It depends on what you mean by "taking silver out." If you asked the IRA trustee to distribute silver to you in-kind, the value of the distribution for tax purposes would be the value of the silver on the date of the distribution and would satisfy an amount of your IRA RMD equal to that value. You should receive a code-7 Form 1099-R reporting a regular distribution with that value shown in box 1 and box 2a and the IRA/SEP/SIMPLE box marked.
If this does not describe the transaction and you somehow physically took the silver out of its place of storage yourself, that would seem to be a prohibited transaction, although I don't see how an IRA trustee would allow you to do so.