Since Barbarians are thematically supposed to be wary of magic use, they should get an alignment penalty whenever they use magic. This could just be for casting spells, or it could be made harder (but more relevant since Barbarians tend to do very little spellcasting anyway) by also assessing the penalty when using obviously magic items like wands or scrolls.