3 ms·
Sure but you could add pseudo random noise instead and get the same behavior while retaining determinism.
by sebzim4500 2y ago
Sure but you could add pseudo random noise instead and get the same behavior while retaining determinism.
- refulgentis 2y agoTemperature is at ~1.2 in this thread, here's some 0.0: - Yes, temperature 0.0 is less creative. - Injecting pseudo-random noise to get deterministic creative outputs is "not even wrong", in the Wolfgang Pauli sense. It's fixing something that isn't broken, with something that can't fix it, that if it could, would be replicating the original behavior - more simply, it's proposing non-deterministic determinism. - Temperature 0.0, in practice, is an LLM. There aren't emergent phenomena, in the sense "emergent phenomena" is used with LLMs, missing. Many, many, many, applications use this. - In simplistic scenarios, on very small models, 0.0 could get stuck literally repeating the same token. - There's a whole other layer of ex. repeat penalties/frequency penalties and such that are used during inference to limit this. Only OpenAI and llama.cpp expose repeat/frequency. - Temperature 0.0 is still non-deterministic on ex. OpenAI, though substantially the same, and even the same most of the time. It's hard to notice differences. (Reproducible builds require extra engineering effort, the same way ensuring temperature = 0.0 is truly deterministic requires engineering effort.) - Pedantically, only temperature 0.0 at the same seed (initial state) is deterministic.
- deleted 2y ago[deleted]