4 ms·
They are deterministic. Random seeding makes them not. But thats a feature.
by holoduke 1y ago
They are deterministic. Random seeding makes them not. But thats a feature.
- xienze 1y agoDoesn’t changing even one word in your prompt affect the output?
- balder1991 1y agoYes, and completely unpredictably.
- localhost 1y agoeven with t=0 they are stochastic. e.g., non associative nature of floating point operations
- int_19h 1y agoThat is an artifact of implementation. You can absolutely implement it using strict FP. But even if not, any given implementation will still do things in a specific order which can be documented. And then if you're running quantized (including KV cache), there's a lot less floating point involved.