If you a destroy a quest leader’s throne, there is a 1 in (Luck + 1) chance that they become angry, but they won’t expel you from the quest until you attack them.