4 ms·

It's akin to us sending a rocket to space and immediately discovering a wormhole. Sure, there's a lot of science about what's out there, but to discover all thi

by ivape 1y ago

It's akin to us sending a rocket to space and immediately discovering a wormhole. Sure, there's a lot of science about what's out there, but to discover all this in our first few trips to orbit ...

silveraxe93 1y ago
Lemme start by saying this is objectively amazing. But I just really wouldn't call it a breakthrough. We had one breakthrough a couple of years ago with GPT-3, where we found that neural networks / transformers + scale does wonders. Everything else has been a smooth continuous improvement. Compare today's announcement to Genie-2[1] release less than 1 year ago. The speed is insane, but not surprising if you put in context on how fast AI is advancing. Again, nothing _new_. Just absurdly fast continuous progress. [1] - https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/ https://deepmind.google/discover/blog/genie-2-a-large-scale-...
- ducktective 1y ago
  Wasn't the model winning gold in IMO result of a breakthrough? I doubt an stochastic parrot can solve math at IMO level...
  - Philpax 1y ago
    As far as we know, it was "just" scale on depth (model capability) and breadth (multiple agents working at the same time).
  - bakuninsbart 1y ago
    Why wouldn't it? I still have to hear one convincing argument how our brain isn't working as a function of probable next best actions. When you look at amoebas work, and animals that are somewhere between them and us in intelligence, and then us, it is a very similar kind of progression we see with current LLMs, from almost no state of the world, to a pretty solid one.
pantalaimon 1y ago
Joscha Bach postulates that what we call consciousness must be something rather simple, an emergent property present in all sufficiently complex biological organisms. We don't inherit any software, so cognitive function must bootstrap itself from it's underlying structure alone. https://media.ccc.de/v/38c3-self-models-of-loving-grace https://media.ccc.de/v/38c3-self-models-of-loving-grace
- glenstein 1y ago
  >We don't inherit any software, so cognitive function must bootstrap itself from it's underlying structure alone. Hardware and software, as metaphors applied to biology, I think are better understood as a continuum than a binary, and if we don't inherit any software (is that true?), we at least inherit assembly code.
  - pantalaimon 1y ago
    > we don't inherit any software (is that true?), we at least inherit assembly code To stay with the metaphor, DNA could be rather understood as firmware that runs on the cell. What I mean with software is the 'mind' that runs on a collection of cells. Things like language, thoughts and ideas. There is also a second level of software that runs not on a single mind alone, but collection of minds, to form cliques or a societies. But this is not encoded in genes, but in memes.
    - glenstein 1y ago
      I think we have some notion of a proto-grammar or ability to linguistically conceptualize, probably at the level of some primordial conceptual units that are more fundamental than language, thoughts and ideas in the concrete forms we generally understand them to have. I think it's like Chomsky said, that we don't learn this infrastructure for understanding language any more than a bird "learns" their feathers. But I might be losing track of what you're suggesting is software in the metaphor. I think I'm broadly on board with your characterization of DNA, the mind and memes generally though.

by ivape 1y ago

It's akin to us sending a rocket to space and immediately discovering a wormhole. Sure, there's a lot of science about what's out there, but to discover all this in our first few trips to orbit ...

silveraxe93 1y ago

Lemme start by saying this is objectively amazing. But I just really wouldn't call it a breakthrough. We had one breakthrough a couple of years ago with GPT-3, where we found that neural networks / transformers + scale does wonders. Everything else has been a smooth continuous improvement. Compare today's announcement to Genie-2[1] release less than 1 year ago. The speed is insane, but not surprising if you put in context on how fast AI is advancing. Again, nothing _new_. Just absurdly fast continuous progress. [1] - https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/ https://deepmind.google/discover/blog/genie-2-a-large-scale-...

ducktective 1y ago
Wasn't the model winning gold in IMO result of a breakthrough? I doubt an stochastic parrot can solve math at IMO level...
- Philpax 1y ago
  As far as we know, it was "just" scale on depth (model capability) and breadth (multiple agents working at the same time).
- bakuninsbart 1y ago
  Why wouldn't it? I still have to hear one convincing argument how our brain isn't working as a function of probable next best actions. When you look at amoebas work, and animals that are somewhere between them and us in intelligence, and then us, it is a very similar kind of progression we see with current LLMs, from almost no state of the world, to a pretty solid one.

pantalaimon 1y ago

Joscha Bach postulates that what we call consciousness must be something rather simple, an emergent property present in all sufficiently complex biological organisms. We don't inherit any software, so cognitive function must bootstrap itself from it's underlying structure alone. https://media.ccc.de/v/38c3-self-models-of-loving-grace https://media.ccc.de/v/38c3-self-models-of-loving-grace

glenstein 1y ago
>We don't inherit any software, so cognitive function must bootstrap itself from it's underlying structure alone. Hardware and software, as metaphors applied to biology, I think are better understood as a continuum than a binary, and if we don't inherit any software (is that true?), we at least inherit assembly code.
- pantalaimon 1y ago
  > we don't inherit any software (is that true?), we at least inherit assembly code To stay with the metaphor, DNA could be rather understood as firmware that runs on the cell. What I mean with software is the 'mind' that runs on a collection of cells. Things like language, thoughts and ideas. There is also a second level of software that runs not on a single mind alone, but collection of minds, to form cliques or a societies. But this is not encoded in genes, but in memes.
  - glenstein 1y ago
    I think we have some notion of a proto-grammar or ability to linguistically conceptualize, probably at the level of some primordial conceptual units that are more fundamental than language, thoughts and ideas in the concrete forms we generally understand them to have. I think it's like Chomsky said, that we don't learn this infrastructure for understanding language any more than a bird "learns" their feathers. But I might be losing track of what you're suggesting is software in the metaphor. I think I'm broadly on board with your characterization of DNA, the mind and memes generally though.