Remove the ability of speed potions to always, reliably confer the speed intrinsic. Instead, you need to already have a lot (at least 100 turns) of temporary very fast speed before drinking a potion of speed will give you the intrinsic.

Various ideas to give the bonus from certain sources of speed only as a bonus to movement, not to getting more turns to attack, use items, etc.

• First, to classify: most agree that speed boots, intrinsic speed, and steed speed should be sources that only give extra movement opportunities, while polyform speed should give extra full turns. Less agreement on magical sources of very fast speed (the potion and haste self), but general opinion seems to be that these should also not be limited to movement.
• Add a second pool of movement points that coexists with regular movement points, but can be used ONLY for moving. Speed boots and its fellow sources add to this pool, and any unused points in it are discarded at the end of the player’s turn. When you make a movement, points are deducted from this pool before being deducted from the “core” pool. Note that this approach can cause core points to accumulate to the point where the player can eventually get multiple non-movement actions on their turn.
• Reduce the movement point cost if you are taking a movement action and have sources of movement-only speed. (This might interact oddly if general-purpose sources of speed continue to add movement points as usual.)

Structure player speed on a four-tier system: normal, fast (available intrinsically), very fast (available extrinsically via speed boots), and extremely fast (temporary and available only via consumables).

Quantum mechanics’ hits toggle your intrinsic speed instead of teleporting you if you have teleport control. (You already know your position, so your velocity must become uncertain.)