The cases that could actually make this happen would almost all have the impact to depend on the IEM and amp used. Silver doesn't have a sound any more than a wire gauge or an impedance number has its own sound. They all participate in a circuit that will decide if and where the consequences can reach audible values.
An anecdotal uncontrolled impression really does nothing to validate sound change, how much, how, or that it's audible.
@bigshot isn't giving the nicest of replies, but from an observer's point of view, psychological biases are very likely to cause part, or all of your "sound" feelings. Leaning toward another possibility without some efforts to first disprove biases would be a mistake.