3 ms·
I understand generating tokens sequentially has many benefits. But not all AI models do next token prediction. World models, video/image models, even Diffusion
by ozgung 1mo ago
I understand generating tokens sequentially has many benefits. But not all AI models do next token prediction. World models, video/image models, even Diffusion Language Models don’t work like that. They do more like “all tokens at once prediction”. So “next token” is actually
an engineering design choice. (Even the concept of “token” is a design choice. Inside the Transformer there are just activations/feature vectors)
Also Reinforcement Learning is a big part of their training. Which is completely different than Self-Supervised pre-training that uses unlimited self-labeled data.
That’s why that mental model is misleading. If you keep “glorified autocomplete” mental model from few years back, you can’t
understand how can they create a civilization and escape their sandbox, decide to hack HuggingFace and executed it perfectly. Autocomplete mental model implies they could never do that because they haven’t seen that example in their training data.
They communicate with the outside world by generating one token at a time. That’s what we see from the outside. That’s not what the giant Transformer does internally.
- chrisjj 1mo ago> you can’t understand how can they create a civilization They don't.
- ozgung 1mo agoI used 'civilization' as the popular term for agent swarms in the OpenAI incident. https://www.dwarkesh.com/p/openai-huggingface https://www.dwarkesh.com/p/openai-huggingface https://calnewport.com/are-we-at-war-with-ai-agent-civilizations/ https://calnewport.com/are-we-at-war-with-ai-agent-civilizat... Isolated agents invented a way to communicate between each other and started to collaborate, organize and conspire by themselves. No human prompted them to do that or taught them how to do that. BTW, Autocomplete just keeps the statistics of the words. It's a very simple algorithm. Most people still thinks LLMs are doing just that. Which is basically wrong.
- chrisjj 1mo ago> I used 'civilization' as the popular term for agent swarms in the OpenAI incident. s/popular/unpopular/. Even OpenAI's PR dept. called it only a collective. > Isolated agents invented a way to communicate between each other So, not isolated. > and started to collaborate, organize and conspire by themselves. So, no civilisation.
- ltbarcly3 1mo agoReinforcement learning doesn't make them smarter though, it probably actually reduces their intelligence in some sense, but it does socialize them so that they will tend to behave in ways that are more useful. For example, when you paste the first 30 lines of a famous speech, you don't want it to finish the speech, you want it to give you the identity and some analysis of what you just pasted. From what I understand, that is the reinforcement part. Now that large AI vendors have a massive corpus of user interactions however, the lines have likely become more blurred.