4 ms·

No one had built a nuclear bomb before the Manhattan project either.

Yizahi 2y ago
Theoretical foundation was slowly built over decades before it started though. And correct me if I'm wrong, but calculations that it was feasible were present before the start too. They had to calculate how to do it, what will be the processes, how to construct it and so on, but theoretically scientists knew that this amount of material can start such process. On the other hand not only there is no clear path to AI today (also known as AGI, ASI, SI etc.), but even foundations are largely missing. We are debating what is intelligence, how it works, how to even start simulating it, or construct from scratch.
- Vecr 2y ago
  There are algorithms that should work, they're just galactic[0] or are otherwise expected to use far too much space and time to be practical. [0]: https://en.wikipedia.org/wiki/Galactic_algorithm https://en.wikipedia.org/wiki/Galactic_algorithm
  - ryan93 2y ago
    That wiki article has nothing to do with AI. The whole AI space attracts BS talk
    - Vecr 2y ago
      What do you think AI is? On that one page there's simulated annealing with a logarithmic cooling schedule, Hutter search, and Solomonoff induction, all very much applicable to AI. If you want a fully complete galactic algorithm for AI, look up AIXItl. Edit: actually I'm not sure if AIXItl is technically galactic or just terribly inefficient, but there's been trouble making it faster and more compact.
- logicchains 2y ago
  The theoretical foundation of transformers is well understood; they're able to approximate a very wide family of functions, particularly with chain of thought ( https://arxiv.org/abs/2310.07923 https://arxiv.org/abs/2310.07923 ). Training them on next-token-prediction is essentially training them to compress, and more optimal compression requires a more accurate model of the world, so they're being trained to model the world better and better. However you want to define intelligence, for practical purposes models with better and better models of the world are more and more useful.
  - zeroonetwothree 2y ago
    The disagreement here seems merely to be about what we mean by “AGI”. I think there’s reasons to think current approaches will not achieve it, but also reason to think they will. In any case anyone who is completely sure that we can/can’t achieve AGI is delusional.
zeroonetwothree 2y ago
this is not evidence in favor of your position. We could use this to argue in favor of anything such as “humans will eventually develop time travel” or “we will have cost effective fusion power”. The fact is many things we’ve tried to develop for decades still don’t exist. Nothing is guaranteed
- mitthrowaway2 2y ago
  I'd put decent odds on a $1B research project developing time travel if time travel were an ability that every human child was innately born with. It's never easy to recreate what biology has done, but nature providing an "existence proof" goes a long way towards removing doubt about it being fundamentally possible.
  - zombiwoof 2y ago
    Nature didn’t build intelligence with non biological activity. And we won’t either
    - mitthrowaway2 2y ago
      There's a big difference between "this project is like time travel or cold fusion; it's doubtful whether the laws of physics even permit it" and "this project is like heavier-than-air flight; we know birds do it somehow, but there's no way our crude metal machines will ever match them". I'm confident which of those problems will get solved given, say, a hundred years or so, once people roll up their sleeves and get working on it.
    - vasco 2y ago
      "Biological activity" is just computation with different energy requirements. If science rules the universe we're complex automata, and biologic machines or non-biological machines are just different combinations of atoms that are computing around.
    - vidarh 2y ago
      Unless you have any evidence suggesting that one or more of the variations of the Church-Turing thesis is false, this is closer to a statement of faith than science. Basically, unless you can show humans calculating a non-Turing computable function, the notion that intelligence requires a biological system is an absolutely extraordinary claim. If you were to argue about conscience or subjective experience or something equally woolly, you might have a stronger point, and this does not at all suggest that current-architecture LLMs will necessarily achieve it.

Theoretical foundation was slowly built over decades before it started though. And correct me if I'm wrong, but calculations that it was feasible were present before the start too. They had to calculate how to do it, what will be the processes, how to construct it and so on, but theoretically scientists knew that this amount of material can start such process. On the other hand not only there is no clear path to AI today (also known as AGI, ASI, SI etc.), but even foundations are largely missing. We are debating what is intelligence, how it works, how to even start simulating it, or construct from scratch.

Vecr 2y ago
There are algorithms that should work, they're just galactic[0] or are otherwise expected to use far too much space and time to be practical. [0]: https://en.wikipedia.org/wiki/Galactic_algorithm https://en.wikipedia.org/wiki/Galactic_algorithm
- ryan93 2y ago
  That wiki article has nothing to do with AI. The whole AI space attracts BS talk
  - Vecr 2y ago
    What do you think AI is? On that one page there's simulated annealing with a logarithmic cooling schedule, Hutter search, and Solomonoff induction, all very much applicable to AI. If you want a fully complete galactic algorithm for AI, look up AIXItl. Edit: actually I'm not sure if AIXItl is technically galactic or just terribly inefficient, but there's been trouble making it faster and more compact.
logicchains 2y ago
The theoretical foundation of transformers is well understood; they're able to approximate a very wide family of functions, particularly with chain of thought ( https://arxiv.org/abs/2310.07923 https://arxiv.org/abs/2310.07923 ). Training them on next-token-prediction is essentially training them to compress, and more optimal compression requires a more accurate model of the world, so they're being trained to model the world better and better. However you want to define intelligence, for practical purposes models with better and better models of the world are more and more useful.
- zeroonetwothree 2y ago
  The disagreement here seems merely to be about what we mean by “AGI”. I think there’s reasons to think current approaches will not achieve it, but also reason to think they will. In any case anyone who is completely sure that we can/can’t achieve AGI is delusional.

zeroonetwothree 2y ago

this is not evidence in favor of your position. We could use this to argue in favor of anything such as “humans will eventually develop time travel” or “we will have cost effective fusion power”. The fact is many things we’ve tried to develop for decades still don’t exist. Nothing is guaranteed

mitthrowaway2 2y ago
I'd put decent odds on a $1B research project developing time travel if time travel were an ability that every human child was innately born with. It's never easy to recreate what biology has done, but nature providing an "existence proof" goes a long way towards removing doubt about it being fundamentally possible.
- zombiwoof 2y ago
  Nature didn’t build intelligence with non biological activity. And we won’t either
  - mitthrowaway2 2y ago
    There's a big difference between "this project is like time travel or cold fusion; it's doubtful whether the laws of physics even permit it" and "this project is like heavier-than-air flight; we know birds do it somehow, but there's no way our crude metal machines will ever match them". I'm confident which of those problems will get solved given, say, a hundred years or so, once people roll up their sleeves and get working on it.
  - vasco 2y ago
    "Biological activity" is just computation with different energy requirements. If science rules the universe we're complex automata, and biologic machines or non-biological machines are just different combinations of atoms that are computing around.
  - vidarh 2y ago
    Unless you have any evidence suggesting that one or more of the variations of the Church-Turing thesis is false, this is closer to a statement of faith than science. Basically, unless you can show humans calculating a non-Turing computable function, the notion that intelligence requires a biological system is an absolutely extraordinary claim. If you were to argue about conscience or subjective experience or something equally woolly, you might have a stronger point, and this does not at all suggest that current-architecture LLMs will necessarily achieve it.