If the metal spellcasting penalty were to be removed entirely for one armor slot, it should be the boots slot, because your feet are very distant from the magical casting part of your body.