6 ms·

To me, the diffusion-based approach "feels" more akin to whats going on in an animal brain than the token-at-a-time approach of the in-vogue LLMs. Speaking for

by kibwen 1y ago

To me, the diffusion-based approach "feels" more akin to whats going on in an animal brain than the token-at-a-time approach of the in-vogue LLMs. Speaking for myself, I don't generate words one a time based on previously spoken words; I start by having some fuzzy idea in my head and the challenge is in serializing it into language coherently.

cube2222 1y ago
I will very often write a message on slack, only to then edit it 5 times… Now I always feel like a diffusion model when I do that.
- djmips 1y ago
  Coding feels like that to me as well.
aabhay 1y ago
The fact that you’re cognitively aware is evidence that this is nowhere near diffusion. More like rumination or thinking tokens, if we absolutely had to find a present day LLM metaphor
crubier 1y ago
You 100% do pronounce or write words one at a time sequentially. But before starting your sentence, you internally formulate the gist of the sentence you're going to say. Which is exactly what happens in LLMs latent space too before they start outputting the first token.
- froobius 1y ago
  (Just to expand on that, it's true not just the for the first token. There's a lot of computation, including potentially planning ahead, before each token outputted.) That's why saying "it's just predicting the next word", is a misguided take.
- smokel 1y ago
  For most serious texts I start with a tree outline, before I engage my literary skills.
- taeric 1y ago
  I'm curious what makes you so confident on this? I confess I expect that people are often far more cognizant of the last thing that the they want to say when they start? I don't think you do a random walk through the words of a sentence as you conceive it. But it is hard not to think people don't center themes and moods in their mind as they compose their thoughts into sentences. Similarly, have you ever looked into how actors learn their lines? It is often in a way that is a lot closer to a diffusion than token at a time.
  - CaptainOfCoit 1y ago
    I think there is a wide range of ways to "turn something in the head into words", and sometimes you use the "this is the final point, work towards it" approach and sometimes you use the "not sure what will happen, lets just start talking and go wherever". Different approaches have different tradeoffs, and of course different people have different defaults. I can confess to not always knowing where I'll end up when I start talking. Similarly, not every time I open my mouth it's just to start but sometimes I do have a goal and conclusion.
  - refulgentis 1y ago
    It's just too far of an analogy, it starts in the familiar SWE tarpit of human brain = lim(n matmuls) as n => infinity. Then, glorifies wrestling in said tarpit: how do people actually compose sentences? Is an LLM thinking or writing? Can you look into how actors memorize lines before responding? Error beyond the tarpit is, these are all ineffable questions that assume a singular answer to an underspecified question across many bags of sentient meat. Taking a step back to the start, we're wondering: Do LLMs plan for token N + X, while purely working to output token N? TL;DR: yes. via https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language.... Clear quick example they have is, ask it to write a poem, get state at end of line 1, scramble the feature that looks ahead to end of line 2's rhyme.
    - jsrozner 1y ago
      Let's just not call it planning. In order to model poetry autoregressively, you're going to need a variable that captures rhyme scheme. At the point where you've ended the first line, the model needs to keep track of the rhyme that was used, just like it does for something like coreference resolution. I don't think that the mentioned paper shows that the model engages in a preplanning phase in which it plans the rhyme that will come. In fact such would be impossible. Model state is present only in so-far-generated text. It is only after the model has found itself in a poetry generating context and has also selected the first line-ending word, that a rhyme scheme "emerges" as a variable. (Now yes, as you increase the posterior probability of 'being in a poem' given context so far, you would expect that you also increase the probability of the rhyme-scheme variable's existing.)
- pessimizer 1y ago
  Like most people I jump back and forth when I speak, disclaiming, correcting, and appending to previous utterances. I do this even more when I write, eradicating entire sentences and even the ideas they contain, within paragraphs that which by the time they were finished the sentence seemed unnecessary or inconsistent. I did it multiple times while writing this comment, and it is only four sentences. The previous sentence once said "two sentences," and after I added this statement it was changed to "four sentences."
- NoMoreNicksLeft 1y ago
  >You 100% do pronounce or write words one at a time sequentially. It's statements like these that make me wonder if I am the same species as everyone else. Quite often, I've picked adjectives and idioms first, and then fill in around them to form sentences. Often because there is some pun or wordplay, or just something that has a nice ring to it, and I want to lead my words in that direction. If you're only choosing them one at a time and sequentially, have you ever considered that you might just be a dimwit? It's not like you don't see this happening all around you in others. Sure you can't read minds, but have you never once watched someone copyedit something they've written, where they move phrases and sentences around, where they switch out words for synonyms, and so on? There are at least dozens of fictional scenes in popular media, you must have seen one. You have to have noticed hints at some point in your life that this occurs. Please. Just tell me that you spoke hastily to score internet argument points, and that you don't believe this thing you've said.
  - crubier 1y ago
    Are you able to pronounce multiple words in superposition at the same time? Are you able to write multiple words in superposition? Can you read the following sentence: "HWeolrllod!" Clearly communication is sequential. LLMs are not more sequential than your vocal chords or your hand writing. They also plan ahead before writing.
  - stevenhuang 1y ago
    All of that can can still be seen as a linear sequence of actions from the perspective of human I/O with the environment. What happens in the black box of the human mind to determine the next word to write/say is exactly made irrelevant in this level of abstraction, as regardless how, it would still result in a linear sequence of actions as observed by the environment.
HPsquared 1y ago
Maybe it's two different modes of thinking. I can have thoughts that coalesce from the ether, but also sometimes string a thought together linearly. Brains might be able to do both.
EGreg 1y ago
I feel completely the opposite way. When you speak or do anything, you focus on what you’re going do next. Your next action. And at that moment you are relying on your recent memory, and things you have put in place while doing the overall activity (context). In fact what’s actually missing from AI currently is simultaneous collaboration, like a group of people interacting — it is very 1 on 1 for now. Like human conversations. Diffusion is like looking at a cloud and trying to find a pattern.
- jensenJRhg 1y ago
  [dead]
ma2rten 1y ago
Interpretability research has found that Autoregressive LLMs also plan ahead what they are going to say.
- aidenn0 1y ago
  This seems likely just from the simple fact that they can reliably generate contextually correct sentences in e.g. German Imperfekt.
  - ma2rten 1y ago
    It's actually true on many levels, if you think about is needed for generating syntactically and grammatically correct sentences, coherent text and working code.
    - aidenn0 1y ago
      Just generating syntactically and grammatically correct sentences doesn't need much lookahead; prefixes to sentences that cannot be properly completed are going to be extremely unlikely to be generated.
  - treis 1y ago
    I don't think you're wrong but I don't think your logic holds up here. If you have a literal translation like: I have a hot dog _____ The word in the blank is not necessarily determined when the sentenced is started. Several verbs fit at the end and the LLM doesn't need to know which it's going to pick when it starts. Each word narrows down the possibilities: I - Trillions Have - Billions a - millions hot - thousands dog - dozens _____ - Could be eaten, cooked, thrown, whatever. If it chooses cooked at this point that doesn't necessarily mean that the LLM was going to do that when it chose "I" or "have"
    - aidenn0 1y ago
      That's why I hedged with "seems likely" and added "in context." If this is in the middle of a paragraph, then there are many fewer options to fit in the blank from the very start.
silveraxe93 1y ago
That's why I'm very excited by Gemini diffusion[1]. - [1] https://deepmind.google/models/gemini-diffusion/ https://deepmind.google/models/gemini-diffusion/
dudu24 1y ago
That is not contrary to token-at-a-time approach.
tripplyons 1y ago
Here's a blog post I liked that explains a connection: https://sander.ai/2024/09/02/spectral-autoregression.html https://sander.ai/2024/09/02/spectral-autoregression.html They call diffusion a form of "spectral autoregression", because it tends to first predict lower frequency features, and later predict higher frequency features.
deleted 1y ago
[deleted]
sailingparrot 1y ago
> the token-at-a-time approach of the in-vogue LLMs. Speaking for myself, I don't generate words one a time based on previously spoken words Autoregressive LLMs don't do that either actually. Sure with one forward pass you only get one token at a time, but looking at what is happening in the latent space there are clear signs of long term planning and reasoning that go beyond just the next token. So I don't think it's necessarily more or less similar to us than diffusion, we do say one word at a time sequentially, even if we have the bigger picture in mind.
- wizzwizz4 1y ago
  If a process is necessary for performing a task, (sufficiently-large) neural networks trained on that task will approximate that process. That doesn't mean they're doing it anything resembling efficiently, or that a different architecture / algorithm wouldn't produce a better result.
  - jama211 1y ago
    It also doesn’t mean they’re doing it inefficiently.
    - pinkmuffinere 1y ago
      I read this to mean “just because the process doesn’t match the problem, that doesn’t mean it’s inefficient”. But I think it does mean that. I expect we intuitively know that data structures which match the structure of a problem are more efficient than those that don’t. I think the same thing applies here. I realize my argument is hand wavey, i haven’t defined “efficient“ (in space? Time? Energy?), and there are other shortcomings, but I feel this is “good enough” to be convincing
      - wizzwizz4 1y ago
        Example: a list of (key, value) pairs is a perfectly valid way to implement a map, and suffices. However, a more complicated tree structure, perhaps with hashed keys, is usually way more efficient, which is increasingly-noticeable as the number of pairs stored in the map grows large.
flux3125 1y ago
It feels like a mix of both to me, diffusion "chunks" being generated in sequence. As I write this comment, I'm deciding on the next word while also shaping the next sentence, like turning a fuzzy idea into a clear sequence.
naasking 1y ago
> Speaking for myself, I don't generate words one a time based on previously spoken words This is a common but fundamentally a weird assumption people have about neurology where they think that what they consciously perceive has some bearing on what's actually happening at the operational or physical level.
wonnage 1y ago
LLMs are notoriously bad at reflecting on how they work and I feel like humans are probably in the same boat

by kibwen 1y ago

cube2222 1y ago

I will very often write a message on slack, only to then edit it 5 times… Now I always feel like a diffusion model when I do that.

djmips 1y ago
Coding feels like that to me as well.

aabhay 1y ago

The fact that you’re cognitively aware is evidence that this is nowhere near diffusion. More like rumination or thinking tokens, if we absolutely had to find a present day LLM metaphor

crubier 1y ago

You 100% do pronounce or write words one at a time sequentially. But before starting your sentence, you internally formulate the gist of the sentence you're going to say. Which is exactly what happens in LLMs latent space too before they start outputting the first token.

froobius 1y ago
(Just to expand on that, it's true not just the for the first token. There's a lot of computation, including potentially planning ahead, before each token outputted.) That's why saying "it's just predicting the next word", is a misguided take.
smokel 1y ago
For most serious texts I start with a tree outline, before I engage my literary skills.
taeric 1y ago
I'm curious what makes you so confident on this? I confess I expect that people are often far more cognizant of the last thing that the they want to say when they start? I don't think you do a random walk through the words of a sentence as you conceive it. But it is hard not to think people don't center themes and moods in their mind as they compose their thoughts into sentences. Similarly, have you ever looked into how actors learn their lines? It is often in a way that is a lot closer to a diffusion than token at a time.
- CaptainOfCoit 1y ago
  I think there is a wide range of ways to "turn something in the head into words", and sometimes you use the "this is the final point, work towards it" approach and sometimes you use the "not sure what will happen, lets just start talking and go wherever". Different approaches have different tradeoffs, and of course different people have different defaults. I can confess to not always knowing where I'll end up when I start talking. Similarly, not every time I open my mouth it's just to start but sometimes I do have a goal and conclusion.
- refulgentis 1y ago
  It's just too far of an analogy, it starts in the familiar SWE tarpit of human brain = lim(n matmuls) as n => infinity. Then, glorifies wrestling in said tarpit: how do people actually compose sentences? Is an LLM thinking or writing? Can you look into how actors memorize lines before responding? Error beyond the tarpit is, these are all ineffable questions that assume a singular answer to an underspecified question across many bags of sentient meat. Taking a step back to the start, we're wondering: Do LLMs plan for token N + X, while purely working to output token N? TL;DR: yes. via https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language.... Clear quick example they have is, ask it to write a poem, get state at end of line 1, scramble the feature that looks ahead to end of line 2's rhyme.
  - jsrozner 1y ago
    Let's just not call it planning. In order to model poetry autoregressively, you're going to need a variable that captures rhyme scheme. At the point where you've ended the first line, the model needs to keep track of the rhyme that was used, just like it does for something like coreference resolution. I don't think that the mentioned paper shows that the model engages in a preplanning phase in which it plans the rhyme that will come. In fact such would be impossible. Model state is present only in so-far-generated text. It is only after the model has found itself in a poetry generating context and has also selected the first line-ending word, that a rhyme scheme "emerges" as a variable. (Now yes, as you increase the posterior probability of 'being in a poem' given context so far, you would expect that you also increase the probability of the rhyme-scheme variable's existing.)
pessimizer 1y ago
Like most people I jump back and forth when I speak, disclaiming, correcting, and appending to previous utterances. I do this even more when I write, eradicating entire sentences and even the ideas they contain, within paragraphs that which by the time they were finished the sentence seemed unnecessary or inconsistent. I did it multiple times while writing this comment, and it is only four sentences. The previous sentence once said "two sentences," and after I added this statement it was changed to "four sentences."
NoMoreNicksLeft 1y ago
>You 100% do pronounce or write words one at a time sequentially. It's statements like these that make me wonder if I am the same species as everyone else. Quite often, I've picked adjectives and idioms first, and then fill in around them to form sentences. Often because there is some pun or wordplay, or just something that has a nice ring to it, and I want to lead my words in that direction. If you're only choosing them one at a time and sequentially, have you ever considered that you might just be a dimwit? It's not like you don't see this happening all around you in others. Sure you can't read minds, but have you never once watched someone copyedit something they've written, where they move phrases and sentences around, where they switch out words for synonyms, and so on? There are at least dozens of fictional scenes in popular media, you must have seen one. You have to have noticed hints at some point in your life that this occurs. Please. Just tell me that you spoke hastily to score internet argument points, and that you don't believe this thing you've said.
- crubier 1y ago
  Are you able to pronounce multiple words in superposition at the same time? Are you able to write multiple words in superposition? Can you read the following sentence: "HWeolrllod!" Clearly communication is sequential. LLMs are not more sequential than your vocal chords or your hand writing. They also plan ahead before writing.
- stevenhuang 1y ago
  All of that can can still be seen as a linear sequence of actions from the perspective of human I/O with the environment. What happens in the black box of the human mind to determine the next word to write/say is exactly made irrelevant in this level of abstraction, as regardless how, it would still result in a linear sequence of actions as observed by the environment.

HPsquared 1y ago

Maybe it's two different modes of thinking. I can have thoughts that coalesce from the ether, but also sometimes string a thought together linearly. Brains might be able to do both.

EGreg 1y ago

I feel completely the opposite way. When you speak or do anything, you focus on what you’re going do next. Your next action. And at that moment you are relying on your recent memory, and things you have put in place while doing the overall activity (context). In fact what’s actually missing from AI currently is simultaneous collaboration, like a group of people interacting — it is very 1 on 1 for now. Like human conversations. Diffusion is like looking at a cloud and trying to find a pattern.

jensenJRhg 1y ago
[dead]

ma2rten 1y ago

Interpretability research has found that Autoregressive LLMs also plan ahead what they are going to say.

aidenn0 1y ago
This seems likely just from the simple fact that they can reliably generate contextually correct sentences in e.g. German Imperfekt.
- ma2rten 1y ago
  It's actually true on many levels, if you think about is needed for generating syntactically and grammatically correct sentences, coherent text and working code.
  - aidenn0 1y ago
    Just generating syntactically and grammatically correct sentences doesn't need much lookahead; prefixes to sentences that cannot be properly completed are going to be extremely unlikely to be generated.
- treis 1y ago
  I don't think you're wrong but I don't think your logic holds up here. If you have a literal translation like: I have a hot dog _____ The word in the blank is not necessarily determined when the sentenced is started. Several verbs fit at the end and the LLM doesn't need to know which it's going to pick when it starts. Each word narrows down the possibilities: I - Trillions Have - Billions a - millions hot - thousands dog - dozens _____ - Could be eaten, cooked, thrown, whatever. If it chooses cooked at this point that doesn't necessarily mean that the LLM was going to do that when it chose "I" or "have"
  - aidenn0 1y ago
    That's why I hedged with "seems likely" and added "in context." If this is in the middle of a paragraph, then there are many fewer options to fit in the blank from the very start.

silveraxe93 1y ago

That's why I'm very excited by Gemini diffusion[1]. - [1] https://deepmind.google/models/gemini-diffusion/ https://deepmind.google/models/gemini-diffusion/

dudu24 1y ago

That is not contrary to token-at-a-time approach.

tripplyons 1y ago

Here's a blog post I liked that explains a connection: https://sander.ai/2024/09/02/spectral-autoregression.html https://sander.ai/2024/09/02/spectral-autoregression.html They call diffusion a form of "spectral autoregression", because it tends to first predict lower frequency features, and later predict higher frequency features.

deleted 1y ago

[deleted]

sailingparrot 1y ago

> the token-at-a-time approach of the in-vogue LLMs. Speaking for myself, I don't generate words one a time based on previously spoken words Autoregressive LLMs don't do that either actually. Sure with one forward pass you only get one token at a time, but looking at what is happening in the latent space there are clear signs of long term planning and reasoning that go beyond just the next token. So I don't think it's necessarily more or less similar to us than diffusion, we do say one word at a time sequentially, even if we have the bigger picture in mind.

wizzwizz4 1y ago
If a process is necessary for performing a task, (sufficiently-large) neural networks trained on that task will approximate that process. That doesn't mean they're doing it anything resembling efficiently, or that a different architecture / algorithm wouldn't produce a better result.
- jama211 1y ago
  It also doesn’t mean they’re doing it inefficiently.
  - pinkmuffinere 1y ago
    I read this to mean “just because the process doesn’t match the problem, that doesn’t mean it’s inefficient”. But I think it does mean that. I expect we intuitively know that data structures which match the structure of a problem are more efficient than those that don’t. I think the same thing applies here. I realize my argument is hand wavey, i haven’t defined “efficient“ (in space? Time? Energy?), and there are other shortcomings, but I feel this is “good enough” to be convincing
    - wizzwizz4 1y ago
      Example: a list of (key, value) pairs is a perfectly valid way to implement a map, and suffices. However, a more complicated tree structure, perhaps with hashed keys, is usually way more efficient, which is increasingly-noticeable as the number of pairs stored in the map grows large.

flux3125 1y ago

It feels like a mix of both to me, diffusion "chunks" being generated in sequence. As I write this comment, I'm deciding on the next word while also shaping the next sentence, like turning a fuzzy idea into a clear sequence.

naasking 1y ago

> Speaking for myself, I don't generate words one a time based on previously spoken words This is a common but fundamentally a weird assumption people have about neurology where they think that what they consciously perceive has some bearing on what's actually happening at the operational or physical level.

wonnage 1y ago

LLMs are notoriously bad at reflecting on how they work and I feel like humans are probably in the same boat