This is in fact exactly what we (https://www.turbinelabs.io https://www.turbinelabs.io) do. With enough proxy work you can tee some traffic to version Y and obs

This is in fact exactly what we (https://www.turbinelabs.io https://www.turbinelabs.io) do. With enough proxy work you can tee some traffic to version Y and observe success rate/latency, then discard the response (or compare it with version X's response), at least for idempotent requests.