4 ms·

Sometimes I really can't tell if these people are serious or not. They seem to believe LLM is some mystical nature formation, or an device made by aliens. Espec

Sometimes I really can't tell if these people are serious or not. They seem to believe LLM is some mystical nature formation, or an device made by aliens. Especially this: > * When we give the LLM a prompt, it simulates every possible entity consistent with the prompt.

MontyCarloHall 4y ago
Right? The whole piece reads like a Sokal Affair redux [0]. [0] https://en.m.wikipedia.org/wiki/Sokal_affair https://en.m.wikipedia.org/wiki/Sokal_affair
- andybak 4y ago
  I don't think it's the same kind of thing at all. Bullshit can come in a wide range of strengths and flavours. This has an interesting core with a whiff of bullshit.
mitthrowaway2 4y ago
Would you object if the sentence read "it approximates simulating every possible entity consistent with the prompt"?
- attemptone 4y ago
  Not OP, but I see the also in problem with 'every possible entity'. If you formulate it like that the prompt is decoupled from the LLM capabilities and can be anything. And if you restrict the prompt to cover only what the LLM understands the sentence becomes trivial. Train a LLM with ASCII and try to get it to simulate anything that is outside of that (ancient sumerian script for example). If you only input ASCII it can generate every possible output in ASCII, most with very low probability but still. After writing this, I'm not even sure what 'simulating' means in this context.
  - remexre 4y ago
    "Every possible entity consistent with the distribution of input data it's been trained with," perhaps? Simulating as in, having equivalent (or "similar enough") input-output behavior, I'd assume.
    - galaxyLogic 4y ago
      "Simulating" has a clear definition, but in this case what is it simulating? "Text-generating entities"? What are these text-generating entities it is (supposedly) simulating? Can you tell me where I can find one? Is it a person like me who writes this reply? So is it trying to simulate me personally? Or are you thinking that it is simulating the aggregated behavior of all humans whose text-outputs are stored on the internet? Are we saying it is simulating the combined input-output -behavior of all humans whose writings appear on the internet? But does such an "entity" exist and does it have behavior? I write this post and you answer. It is you who answers, not some mythical text-generator-entity that is responsible for all texts on the internet. There is no such entity is there? It does not make sense to say that we are simulating the behavior of some non-existent entity. Non-existent entities do not have behavior, therefore we can not simulate them.
      - mitthrowaway2 4y ago
        There clearly exists a computable function that is a good enough approximation of "galaxyLogic's reply to remexre's comment" that it might be hard to for me tell whether the output was generated by the human brain or by an LLM. That function might indeed end up reproducing the same steps that your brain follows in constructing a reply. (Just speaking hypothetically here). While we understand LLMs, we don't understand the human brain, and in particular I don't think we've yet proven that human brains don't contain embedded routines that are similar to LLMs. Someone with your particular writing style might be one, of several, simulations that are approximated within the LLM. Just like I can have it respond in the style of Spock from Star Trek.
        galaxyLogic 4y ago
ineptech 4y ago
I think you're taking it way too literally. The author is proposing a model that [allegedly] does a good job of explaining why we see unexpected behavior, not a description of what they think is happening. I think charitable reading would require you to mentally insert a lot of qualifiers, e.g. "When we give the LLM a prompt, it [behaves like something that] simulates every possible entity consistent with the prompt." That's certainly how I read it, anyway. edit to add: this is similar to how people discussing evolutionary biology will often use "evolution wants to..." as shorthand for something like "evolution, which obviously cannot want things due to being a process and not an entity, nevertheless can be accurately modeled as an entity that wants to...". Someone will invariably come along in the comments and say, "Nonsense, how can evolution 'want' anything? You must have failed Bio 101!"
- deleted 4y ago
  [deleted]
- sdwr 4y ago
  As with all these articles, the title and main premise are misguided and/or imaginary, but there are some worthwhile nuggets. The superposition of possible attitudes is a good one. Even if that's not the way LLMs "actually" work, it's descriptive of the possibility space from our perspective. And the dive into narrative theory + the stickiness of opposites is nice. Narratives have their own momentum in a "stone soup" kind of way - everyone who hears it participates and adds fuel to the fire. Even rejecting the narrative gives it validity in a price anchoring / overton window way.
- donnowhy 4y ago
  > The author is proposing a model that [allegedly] does a good job of explaining why we see unexpected behavior I agree, and go even further: models that explain behavior are all we have ever had. it's all only "models that explain this or that" all the way to the 'bottom'. To suppose we can really directly access the "the real objective truth of what's happening" is to ignore the way in which we connect with the "real objective truth"; the same as fish who ignore the ocean. to argue about what is really happening is to argue about which words to use to describe what is really happening without noticing the nature of languages/words and frameworks or 'systems of thought' which we are using to argue (and indeed, are arguing about) all this summed up by this quote from about about the pedagogy programing languages: "Sometimes the truest things can only be said in fiction"
assbuttbuttass 4y ago
> When we give the LLM a prompt, it simulates every possible entity consistent with the prompt. This is just Bayes' rule. The probability of an LLM generating any particluar output is the sum over every possible entity of the probability of seeing that entity multiplied by the chance that entity would generate the output.

MontyCarloHall 4y ago

Right? The whole piece reads like a Sokal Affair redux [0]. [0] https://en.m.wikipedia.org/wiki/Sokal_affair https://en.m.wikipedia.org/wiki/Sokal_affair

andybak 4y ago
I don't think it's the same kind of thing at all. Bullshit can come in a wide range of strengths and flavours. This has an interesting core with a whiff of bullshit.

mitthrowaway2 4y ago

Would you object if the sentence read "it approximates simulating every possible entity consistent with the prompt"?

attemptone 4y ago
Not OP, but I see the also in problem with 'every possible entity'. If you formulate it like that the prompt is decoupled from the LLM capabilities and can be anything. And if you restrict the prompt to cover only what the LLM understands the sentence becomes trivial. Train a LLM with ASCII and try to get it to simulate anything that is outside of that (ancient sumerian script for example). If you only input ASCII it can generate every possible output in ASCII, most with very low probability but still. After writing this, I'm not even sure what 'simulating' means in this context.
- remexre 4y ago
  "Every possible entity consistent with the distribution of input data it's been trained with," perhaps? Simulating as in, having equivalent (or "similar enough") input-output behavior, I'd assume.
  - galaxyLogic 4y ago
    "Simulating" has a clear definition, but in this case what is it simulating? "Text-generating entities"? What are these text-generating entities it is (supposedly) simulating? Can you tell me where I can find one? Is it a person like me who writes this reply? So is it trying to simulate me personally? Or are you thinking that it is simulating the aggregated behavior of all humans whose text-outputs are stored on the internet? Are we saying it is simulating the combined input-output -behavior of all humans whose writings appear on the internet? But does such an "entity" exist and does it have behavior? I write this post and you answer. It is you who answers, not some mythical text-generator-entity that is responsible for all texts on the internet. There is no such entity is there? It does not make sense to say that we are simulating the behavior of some non-existent entity. Non-existent entities do not have behavior, therefore we can not simulate them.
    - mitthrowaway2 4y ago
      There clearly exists a computable function that is a good enough approximation of "galaxyLogic's reply to remexre's comment" that it might be hard to for me tell whether the output was generated by the human brain or by an LLM. That function might indeed end up reproducing the same steps that your brain follows in constructing a reply. (Just speaking hypothetically here). While we understand LLMs, we don't understand the human brain, and in particular I don't think we've yet proven that human brains don't contain embedded routines that are similar to LLMs. Someone with your particular writing style might be one, of several, simulations that are approximated within the LLM. Just like I can have it respond in the style of Spock from Star Trek.
      - galaxyLogic 4y ago

ineptech 4y ago

I think you're taking it way too literally. The author is proposing a model that [allegedly] does a good job of explaining why we see unexpected behavior, not a description of what they think is happening. I think charitable reading would require you to mentally insert a lot of qualifiers, e.g. "When we give the LLM a prompt, it [behaves like something that] simulates every possible entity consistent with the prompt." That's certainly how I read it, anyway. edit to add: this is similar to how people discussing evolutionary biology will often use "evolution wants to..." as shorthand for something like "evolution, which obviously cannot want things due to being a process and not an entity, nevertheless can be accurately modeled as an entity that wants to...". Someone will invariably come along in the comments and say, "Nonsense, how can evolution 'want' anything? You must have failed Bio 101!"

deleted 4y ago
[deleted]
sdwr 4y ago
As with all these articles, the title and main premise are misguided and/or imaginary, but there are some worthwhile nuggets. The superposition of possible attitudes is a good one. Even if that's not the way LLMs "actually" work, it's descriptive of the possibility space from our perspective. And the dive into narrative theory + the stickiness of opposites is nice. Narratives have their own momentum in a "stone soup" kind of way - everyone who hears it participates and adds fuel to the fire. Even rejecting the narrative gives it validity in a price anchoring / overton window way.
donnowhy 4y ago
> The author is proposing a model that [allegedly] does a good job of explaining why we see unexpected behavior I agree, and go even further: models that explain behavior are all we have ever had. it's all only "models that explain this or that" all the way to the 'bottom'. To suppose we can really directly access the "the real objective truth of what's happening" is to ignore the way in which we connect with the "real objective truth"; the same as fish who ignore the ocean. to argue about what is really happening is to argue about which words to use to describe what is really happening without noticing the nature of languages/words and frameworks or 'systems of thought' which we are using to argue (and indeed, are arguing about) all this summed up by this quote from about about the pedagogy programing languages: "Sometimes the truest things can only be said in fiction"

assbuttbuttass 4y ago

> When we give the LLM a prompt, it simulates every possible entity consistent with the prompt. This is just Bayes' rule. The probability of an LLM generating any particluar output is the sum over every possible entity of the probability of seeing that entity multiplied by the chance that entity would generate the output.