3 ms·
The LLM isn't being perfectly reset. It chooses words randomly; internally it should be slightly different every time. That's the whole point of temperature.
by hwillis 2y ago
The LLM isn't being perfectly reset. It chooses words randomly; internally it should be slightly different every time. That's the whole point of temperature.
- tempusalaria 2y agoTemperature has nothing to do with internals. Temperature is purely to do with how the logits outputted by the network are transformed into probabilities, which is completely deterministic and not learned. In fact, temperature makes it impossible for LLMs to simulate this kind of probability. As a calibrated 80-20 split at a certain low temperature would be a different split with some other temperature.
- gudd0g 2y ago[dead]