2 ms·
I used that description some months ago and I think you are the first person I see who put it the same way. The only gotcha with this is that they are theoreti
by SkyBelow 10d ago
I used that description some months ago and I think you are the first person I see who put it the same way.
The only gotcha with this is that they are theoretically deterministic, but rarely in practice.
A few examples:
- Harness specific settings that user can't control (anything from timestamp to prng seeding.
- Batching requests in a way that leads to a single request being processed different depending upon the batch (say MoE where your first choice expert is assigned to someone else's token so you go to your second choice vs a batch where you get your first choice).
- Graphics card itself carrying out floating point arithmetic in slightly different orders leading to floating point non associativity causing different outputs.
But all of these can be controlled for (at some cost) and the model can be ran deterministically.
For the average user, it might as well be non-deterministic, but when considering theoretical capabilities, chaotic deterministic system seems the better description.