3 ms·
The impressive part is that it is low latency enough to serve high quality answers at game speed through the model instead of a pre generated ad-hoc machine.
by someguynamedq 26d ago
The impressive part is that it is low latency enough to serve high quality answers at game speed through the model instead of a pre generated ad-hoc machine.
- bigglebear 25d agoA pre-generated machine can serve the answers in <1ms. It's a far better strategy.