Zapping a wand requires touching it, so unless you are wearing gloves, this causes you to incur any effects, such as silver damage, from handling something of its material. Conversely, monsters usually won’t use a wand that will hurt them (unless they’re wearing gloves), and will incur material effects if they do.