Track the amount of Luck increase you have had from a unicorn. If you steal gems from such a unicorn or kill a unicorn you have gotten Luck from, immediately revert that Luck increase.