Cavemen being illiterate makes a lot of sense, but forcing them to be illiterate is unlikely to make them fun to play when they’re already one of the least fun roles to play. So instead, the role should have mechanics that make them compatible with illiterate and a good choice if one is trying to win an illiterate game in particular. But they can also be prodded towards being illiterate by having them abuse alignment every time they read something (possibly only scrolls and books, not things like fortune cookies).

Take the “Caveman’s role strength should be having stats through the roof” idea to an extended version: not only should they have higher physical stat maximums than pretty much everyone else, but they should also get an increased melee damage bonus, a higher HP maximum, and a higher carrying capacity.

To compensate for this, they could be made illiterate by being incapable of reading anything.

If you are illiterate and receive some mail, the mail daemon will offer to read it to you.

When a mail daemon delivers a scroll of mail and you have kept illiterate conduct so far, it offers to read the message to you rather than just giving you the scroll which you can’t read.

Don’t gift a character a spellbook as a prayer boon if the character is illiterate or certain roles (like cavemen or barbarians), and if a spellbook is given limit its level somehow so that a character doesn’t get a spellbook of much too high level.

Zen or illiterate players get spells for free as a prayer boon where they normally would have been granted the corresponding spellbook.