Remove the ability of speed potions to always, reliably confer the speed intrinsic. Instead, you need to already have a lot (at least 100 turns) of temporary very fast speed before drinking a potion of speed will give you the intrinsic.

Quaffing a potion of speed should exercise dexterity. Possibly only if it is non-cursed.

The wand of speed, zapped at yourself, grants temporary very fast speed and does not grant intrinsic speed. Intrinsic speed can be gained from a non-cursed potion of speed, and a blessed potion will still grant temporary very fast speed. These would be more useful in a system where permanent very fast speed is unobtainable.