4 ms·
>> an LLM is completely opaque And despite that, although they are not like that in practice as there are too many uncontrolled variables, with temperature at
by Betelbuddy 2mo ago
>> an LLM is completely opaque
And despite that, although they are not like that in practice as there are too many uncontrolled variables, with temperature at zero, for the same input they produce always the same reply.
- irishcoffee 2mo agoHa, they sure don’t.
- Betelbuddy 2mo agoThey do. Just train your own LLM, not that difficult, and you will have a more controlled environment and you will see they do.
- chrisjj 2mo ago> with temperature at zero, for the same input they produce always the same reply. Nonsense. https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
- coldtea 2mo agoBS. Run them sequentially on a single core, and without fancy speedups enabled, and they do. The algorithm is determinstic. Any non-determinism present with 0 temperature it's not some mysterious LLM-inherent property, but something that can be seen in any large program taking advantage of multi-core, floating point, and other CPU-based parallelism optimization.
- hashstring 2mo agoExactly, this is correct. People often assume they are not because they can ask the same query to the same model and get differences in output, but wrongly conclude that this is some inherent LLM trait, instead of non-determinism added on top of it because of implementational choices that were made.
- chrisjj 2mo ago> Any non-determinism present with 0 temperature it's not some mysterious LLM-inherent property, Of course it's not mysterious. It is well understood by all who are aware of the fundamental unreliability of all major LLMs in general use today.
- coldtea 2mo agoThe phrasing still makes it sound like it's due to the LLM. It's not. It's do to compiler and CPU, and OS optimizations, and a non-LLM program could suffer the same just as well.
- chrisjj 2mo agoThis unreliability is entirely due to the implentation of the LLM. Yes, any other program implemented equally carelessly could suffer the same. But you'd be hard pressed to find any as unreliable as a typical LLM.
- theptip 2mo agoIf we digitize your brain and reload the checkpoint before you posted, you’ll type this message again in exactly the same way. Just because a black-box system is deterministic, doesn’t mean it’s understood. You couldn’t predict the output the first time around, is the point.