Cavemen being illiterate makes a lot of sense, but forcing them to be illiterate is unlikely to make them fun to play when they’re already one of the least fun roles to play. So instead, the role should have mechanics that make them compatible with illiterate and a good choice if one is trying to win an illiterate game in particular. But they can also be prodded towards being illiterate by having them abuse alignment every time they read something (possibly only scrolls and books, not things like fortune cookies).