4 ms·
It still works the same way other LLMs do, by outputting the probability distribution over the possible completions (The weather is ... (sunny (50%), cloudy (50
by v9v 2mo ago
It still works the same way other LLMs do, by outputting the probability distribution over the possible completions (The weather is ... (sunny (50%), cloudy (50%))). Then the next token is sampled from this probability distribution (in our example the next word could be "sunny" or "cloudy" equally likely), which can result in different outputs every run.
- logicallee 2mo agoCould the model or algorithm be changed to make it deterministic somehow? It could help a lot if there were reproduceable outputs from deterministic baked-in silicon.
- Tuna-Fish 2mo agoYou can make any LLM deterministic by dropping the temperature hyperparameter to zero. This will generally make them suck, though, a little bit of randomness is necessary for proper function.
- fwip 2mo agoYou can also use a fixed seed for your prng. A hash of the input text (up to the current turn) should do.
- fph 2mo agoBut since it's so fast you can just ask it 100 times where Bruce Lee was born, and statistically you'll get the correct answer. We could call it "mixture of idiots". /s