You are right, but it's easier than you think. At my workplace we have anycast without a ton of "tie breakers" so each client may have several backend servers t

You are right, but it's easier than you think. At my workplace we have anycast without a ton of "tie breakers" so each client may have several backend servers that it could reach. But the router itself does do the IP based hashing when there are two valid routes to the same prefix. So yes, there is something doing hashing, but it's not anything special, it's just the usual "equal-cost multipath" behavior on the router.