3 ms·
One effect of this technology being almost totally opaque, non-explainable, and also nondeterministic is that most of the arguments for it (and maybe most again
by lsy 2y ago
One effect of this technology being almost totally opaque, non-explainable, and also nondeterministic is that most of the arguments for it (and maybe most against it?) take on an anecdotal character.
I was surprised to hear Dr. Bubeck, who is in kind of a privileged position wrt OpenAI and obviously an accomplished scientist, essentially saying things like "I tried asking it X and I am pretty sure it's not in the training set, and it worked, therefore I think it understands".
A really big problem with the anecdata approach to proving out AI is alluded to in Dr. Bender's story about the "Everything in the Whole Wide World" museum (for those who didn't watch, Grover the muppet goes to the aforementioned museum, sees many things — but not "everything", then walks through a door labeled "Everything Else" that leads outside). That is, no individual prompt-and-response can be relied on to inform about another prompt-and-response that you haven't yet tried. And with nondeterminism, you can't even rely on that prompt-and-response to remain stable. But a failed prompt also carries the possibility that a tweaked prompt could produce a success. So (as we saw with the twitter post yesterday complaining about LLMs not working well that was both heavily upvoted and heavily contested in the comments), we are in a world where nobody can say much except to provide anecdotes of the thing failing or succeeding or having characteristic Y, which are exercises in narrative construction to support or oppose investment, not principled discussion.
- dghlsakjg 2y agoAre llms truly non-deterministic? My impression was that we inject some randomness intentionally to mimic it, but if you remove that factor you end up with a deterministic chat model.
- CuriouslyC 2y agoThe probability distribution over next tokens given previous tokens is deterministic. The sampling algorithm for that distribution is non-deterministic.
- krallistic 2y agoAnd sampling from a (now fixed) distribution can be made deterministic... So the total generation of text from an LLM can be made fully deterministic. The problem for scientists is that we cant do that in the deployed systems...
- CuriouslyC 2y agoYou can set the temperature to zero in most APIs, which gives deterministic output. The only problem with that is some models produce inferior results with zero temperature, including lots of slop and AI-isms.
- Kamshak 2y agoThere is also unintentional randomness due to the parallelism in inference (e.g. parallel matmuls added together on the GPU). Since it's multiplying floats every operation has rounding drift that accumulates differently depending on the order of operations. So even at temperature 0 you're not getting deterministic outputs
- naveen99 2y agoBecause addition and multiplication are not associative with floats ?
- lsy 2y agoIn addition to Kamshak's note about parallel inference accumulating float errors differently due to order of operations, which makes LLMs theoretically non-deterministic at temperature 0, there is the issue of them being practically non-deterministic as-deployed, not just via temperature but because of inclusion of prior "turns" in context, variations in phrasing of prompts, etc. It's also "non-deterministic" in the sense that if you removed all sources of non-determinism and asked "What is 1+1?" and received the answer "2" deterministically, that doesn't guarantee a correct answer for "What is 1+2?". Ie a variation in the input isn't correlated in a logical way with a variation in the output, which is somewhat fatal for computer programs, where the goal is to generalize a problem across a range of inputs.