3 ms·

I truly think you haven't really thought this through. There's a huge amount of circuitry between the input and the output of the model. How do you know what i

by svara 2y ago

I truly think you haven't really thought this through. There's a huge amount of circuitry between the input and the output of the model. How do you know what it does or doesn't do? Humans brains "just" output the next couple milliseconds of muscle activation, given sensory input and internal state. Edit: Interestingly, this is getting downvotes even though 1) my last sentence is a precise and accurate statement of the state of the art in neuroscience and 2) it is completely isomorphic to what the parent post presented as an argument against current models being AGI. To clarify, I don't believe we're very close to AGI, but parent's argument is just confused.

015a 2y ago
Did you seriously just use the word "isomorphic"? No wonder people believe AI is the next crypto.
- svara 2y ago
  Well, AI clearly is the next crypto, haha. Apologies for the wording but I think you got it and the point stands. I'm not a native speaker and mostly use English in a professional science related setting, that's why I sound like that sometimes. isomorphic - being of identical or similar form, shape, or structure (m-w). Here metaphorically applied to the structure of an argument.
- edouard-harris 2y ago
  In what way was their usage incorrect? They simply said that the brain just predicts next-actions, in response to a statement that an LLM predicts next-tokens. You can believe or disbelieve either of those statements individually, but the claims are isomorphic in the sense that they have the same structure.
  - 015a 2y ago
    Its not that it was used incorrectly: Its that it isn't a word actual humans use, and its one of a handful of dog whistles for "I'm a tech grifter who has at best a tenuous grasp on what I'm talking about but would love more venture capital". The last time I've personally heard it spoken was from Beff Jezos/Guillaume Verdon.
    - NoGravitas 2y ago
      I think we should delve further into that analysis.
    - svara 2y ago
      You know, you can just talk to me about my wording. Where do I meet those gullible venture investors?
HarHarVeryFunny 2y ago
> There's a huge amount of circuitry between the input and the output of the model Yeah - but it's just a stack of transformer layers. No looping, no memory, no self-modification (learning). Also, no magic.
- svara 2y ago
  No looping, but you can unroll loops to a fixed depth and apply the model iteratively. There obviously is memory and learning. Neuroscience hasn't found the magic dust in our brains yet, either. ;)
  - HarHarVeryFunny 2y ago
    Zero memory inside the model from one input (ie token output) to the next (only the KV cache, which is just an optimization). The only "memory" is what the model outputs and therefore gets to re-consume (and even there it's an odd sort of memory since the model itself didn't exactly choose what to output - that's a random top-N sampling). There is no real runtime learning - certainly no weight updates. The weights are all derived from pre-training, and so the runtime model just represents a frozen chunk of learning. Maybe you are thinking of "in-context learning", which doesn't update the weights, but is rather the ability of the model to use whatever is in the context, including having that "reinforced" by repetition. This is all a poor substitute for what an animal does - continuously learning from experience and exploration. The "magic dust" in our brains, relative to LLMs, is just a more advanced and structure architecture, and operational dynamics. e.g. We've got the thalamo-cortical loop, massive amounts of top-down feedback for incremental learning from prediction failure, working memory, innate drives such as curiosity (prediction uncertainty) and boredom to drive exploration and learning, etc, etc. No magic, just architecture.
    - svara 2y ago
      I'm not entirely sure what you're arguing for. Current AI models can still get a lot better, sure. I'm not in the AGI in 3 years camp. But, people in this thread are making philosophically very poor points about why that is supposedly so. It's not "just" sequence prediction, because sequence prediction is the very essence of what the human brain does. Your points on learning and memory are similarly weak word play. Memory means holding some quantity constant over time in the internal state of a model. Learning means being able to update those quantities. LLMs obviously do both. You're probably going to be thinking of all sorts of obvious ways in which LLMs and humans are different. But no one's claiming there's an artificial human. What does exist is increasingly powerful data processing software that progressively encroaches on domains previously thought to be that of humans only. And there may be all sorts of limitations to that, but those (sequences, learning, memory) aren't them.

by svara 2y ago

015a 2y ago

Did you seriously just use the word "isomorphic"? No wonder people believe AI is the next crypto.

svara 2y ago
Well, AI clearly is the next crypto, haha. Apologies for the wording but I think you got it and the point stands. I'm not a native speaker and mostly use English in a professional science related setting, that's why I sound like that sometimes. isomorphic - being of identical or similar form, shape, or structure (m-w). Here metaphorically applied to the structure of an argument.
edouard-harris 2y ago
In what way was their usage incorrect? They simply said that the brain just predicts next-actions, in response to a statement that an LLM predicts next-tokens. You can believe or disbelieve either of those statements individually, but the claims are isomorphic in the sense that they have the same structure.
- 015a 2y ago
  Its not that it was used incorrectly: Its that it isn't a word actual humans use, and its one of a handful of dog whistles for "I'm a tech grifter who has at best a tenuous grasp on what I'm talking about but would love more venture capital". The last time I've personally heard it spoken was from Beff Jezos/Guillaume Verdon.
  - NoGravitas 2y ago
    I think we should delve further into that analysis.
  - svara 2y ago
    You know, you can just talk to me about my wording. Where do I meet those gullible venture investors?

HarHarVeryFunny 2y ago

> There's a huge amount of circuitry between the input and the output of the model Yeah - but it's just a stack of transformer layers. No looping, no memory, no self-modification (learning). Also, no magic.

svara 2y ago
No looping, but you can unroll loops to a fixed depth and apply the model iteratively. There obviously is memory and learning. Neuroscience hasn't found the magic dust in our brains yet, either. ;)
- HarHarVeryFunny 2y ago
  Zero memory inside the model from one input (ie token output) to the next (only the KV cache, which is just an optimization). The only "memory" is what the model outputs and therefore gets to re-consume (and even there it's an odd sort of memory since the model itself didn't exactly choose what to output - that's a random top-N sampling). There is no real runtime learning - certainly no weight updates. The weights are all derived from pre-training, and so the runtime model just represents a frozen chunk of learning. Maybe you are thinking of "in-context learning", which doesn't update the weights, but is rather the ability of the model to use whatever is in the context, including having that "reinforced" by repetition. This is all a poor substitute for what an animal does - continuously learning from experience and exploration. The "magic dust" in our brains, relative to LLMs, is just a more advanced and structure architecture, and operational dynamics. e.g. We've got the thalamo-cortical loop, massive amounts of top-down feedback for incremental learning from prediction failure, working memory, innate drives such as curiosity (prediction uncertainty) and boredom to drive exploration and learning, etc, etc. No magic, just architecture.
  - svara 2y ago
    I'm not entirely sure what you're arguing for. Current AI models can still get a lot better, sure. I'm not in the AGI in 3 years camp. But, people in this thread are making philosophically very poor points about why that is supposedly so. It's not "just" sequence prediction, because sequence prediction is the very essence of what the human brain does. Your points on learning and memory are similarly weak word play. Memory means holding some quantity constant over time in the internal state of a model. Learning means being able to update those quantities. LLMs obviously do both. You're probably going to be thinking of all sorts of obvious ways in which LLMs and humans are different. But no one's claiming there's an artificial human. What does exist is increasingly powerful data processing software that progressively encroaches on domains previously thought to be that of humans only. And there may be all sorts of limitations to that, but those (sequences, learning, memory) aren't them.