You can break and destroy a loadstone in inventory by applying a pickaxe to yourself, or rubbing the pickaxe on the loadstone.