3 ms·
And computers are just lots of really fast logic gates. I think the issue with reducing LLMs to "next word predictors" is that it focuses on one part of the me
by Closi 3y ago
And computers are just lots of really fast logic gates.
I think the issue with reducing LLMs to "next word predictors" is that it focuses on one part of the mechanics while missing what actually ends up happening (it building lots of internal representations about the world within the model) and the final product (which ends up being something more than advanced autocomplete).
Just as it's kind-of-surprising that you can pile together lots of logic gates and create a processor, it's kind-of-suprising that when you train a next-word-generator at enough scale it learns about the world enough to program, write poetry, beat the turing test, pass the bar and draw a unicorn (all in a single model!).
- enterprise_cog 3y agoPoor analogy given logic gates are deterministic while an LLM is not.
- viraptor 3y agoLLM implementation may be deterministic or not. The idea/tech itself does not restrict this in any way.
- baobabKoodaa 3y agoYou can coerce deterministic and reproducible outputs from an LLM
- Closi 3y agoWhat do you think LLMs are made of?
- Grimblewald 3y agopoor question, we know what human brains are made of, doesn't help us understand them all too much.
- RugnirViking 3y agoLLMs are deterministic though? like with 0 temprature you always get the same output (temprature is literally injected randomness because by default they are "too" deterministic)
- enterprise_cog 3y agoA 0 temp LLM is just an autocomplete. Not even spicy.