4 ms·

> Hassabis said that one or two more critical breakthroughs were needed before AGI was reached. But he added: “I wouldn’t be surprised if it happened in the nex

> Hassabis said that one or two more critical breakthroughs were needed before AGI was reached. But he added: “I wouldn’t be surprised if it happened in the next decade. I’m not saying it’s definitely going to happen but I wouldn’t be surprised. You could say about a 50 per cent chance." What was Hassabis saying about hype, again?

ein0p 3y ago
Note: dude knows perfectly well that we are not only very far away from AGI, but that we also haven’t even begun solving some of the major problems with the current approaches. For a more sober take on AGI, read Yann LeCun’s paper on the topic. It’s over a year old, so ancient by the standards of this field, but still accurate https://openreview.net/pdf?id=BZ5a1r-kVsf https://openreview.net/pdf?id=BZ5a1r-kVsf
- mateo1 3y ago
  Nice paper. But my take is this: If this model of operation of the human mind is an emergent property of a simpler network architecture that's scaled up, nothing is stopping us from building a massive version of it and training it. This can be done in 10 years. However if it's a combination of myriads of bespoke mechanisms that emerged evolutionarily then it's going to take a while, because we have to rediscover them through research or find a way to detect and copy them from our brains. For example if there are tons and tons of multi-scale mechanisms (network architectures+molecular dynamics+feedback systems+adaptive cell responses) for each function of the human brain then yeah it's going to take a long time. But if there's only a handful of bespoke systems related to cognition (memory, emotional regulation, motor skills, perception, motivation) then it's very much doable. Nobody thought that you could chat with a simple CNN 15 years ago, but here we are.
  - ein0p 3y ago
    Transformers are not CNNs. Neural nets were the second best model for anything since their inception and it is proven mathematically that they can approximate any conceivable function so as long as it’s continuous. The issue at hand is that we have not a foggiest clue as to what it is we’re trying to approximate, so we’re now hoping that we get lucky again, and something will “emerge” somehow. Hope, however, is not a strategy. There’s no guarantee. If Demis believes what he says, he should put up a bet with a nontrivial amount. He won’t do that - he’s not stupid.
- barfbagginus 3y ago
  Yann is respected in the field, and he has some good ideas in that paper and others. But he not advancing the state of the art on LLMs. Ie, he talked about a world model and actor/critic model, but it's other people who are exploring LLM driven world/actor/critic models. If you're not reading papers published in the last months, you're not going to see how quickly things are improving, compared to a year ago. Here's just one example: the incredible explosion in context length. A year ago GPT4 would hallucinate critical details from a 10 page 5000 word essay. Today, Gemini 1.5 can read a 1000 page 5 million word textbook, and recall facts with 99.5% accuracy. This is already an AGI level feat. Now it just needs adaptors, memory modules. AGI doesn't imply it can self improve. It's just generally intelligent and capable of many tasks. Off the shelf LLMs with accessory modules will be enough to build AGI capability and change the nature of work and knowledge.
  - ein0p 3y ago
    Yes, we’re now pretty good at associative memory. But associative memory is but a small part of the whole picture. Read the paper, clouds might part a little and enthusiasm might subside. I quite literally work on LLMs as my day job. They alone are not going to somehow get us to AGI
    - barfbagginus 3y ago
      Please tell me which sections of the paper you feel suggest that LLMs + extensions cannot achieve AGI. I have read the paper 3 times in the last year, but I'm not seeing it. There's basically no reason you wouldn't be able to implement LeCunn's entire JEPA architecture using LLM analogues. Associative memory is just one of many huge improvements we could discuss And before you object that LeCunn's methods do online learning with energy methods, let me point to RAG systems that also do online learning - and better than anything Lecunn has published. I own a company where I build new computational geometry codes. And I'm increasingly using LLMs to research, plan, implement, test, and debug my codes. Not only that, but I am an autistic person who increasingly uses LLMs to overcome my social and communication difficulties. GPT is far more empathic and constructive with it's suggestions than any friend, therapist, or advisor I have worked with or have access to. So again it already is AGI for the purpose of helping me process my often complex and baffling emotions. Nobody's saying pure LLMs will be AGI. But pure LLMs working together in novel configurations and using specific components already have that capability. Memory, empathy, planning, agency, theory of mind, etc. You name it, I'll show you a paper where LLMs demonstrate ir. GPT4 by itself does these things better than most people. These capabilities will be packaged into off the shelf components. They will be integrated into post-llm systems that merge these capabilities into a system with the most of the capabilities of a conscious human. That'll be AGI. But what we already have is already AGI as well. You're just not giving it credit.

ein0p 3y ago

Note: dude knows perfectly well that we are not only very far away from AGI, but that we also haven’t even begun solving some of the major problems with the current approaches. For a more sober take on AGI, read Yann LeCun’s paper on the topic. It’s over a year old, so ancient by the standards of this field, but still accurate https://openreview.net/pdf?id=BZ5a1r-kVsf https://openreview.net/pdf?id=BZ5a1r-kVsf

mateo1 3y ago
Nice paper. But my take is this: If this model of operation of the human mind is an emergent property of a simpler network architecture that's scaled up, nothing is stopping us from building a massive version of it and training it. This can be done in 10 years. However if it's a combination of myriads of bespoke mechanisms that emerged evolutionarily then it's going to take a while, because we have to rediscover them through research or find a way to detect and copy them from our brains. For example if there are tons and tons of multi-scale mechanisms (network architectures+molecular dynamics+feedback systems+adaptive cell responses) for each function of the human brain then yeah it's going to take a long time. But if there's only a handful of bespoke systems related to cognition (memory, emotional regulation, motor skills, perception, motivation) then it's very much doable. Nobody thought that you could chat with a simple CNN 15 years ago, but here we are.
- ein0p 3y ago
  Transformers are not CNNs. Neural nets were the second best model for anything since their inception and it is proven mathematically that they can approximate any conceivable function so as long as it’s continuous. The issue at hand is that we have not a foggiest clue as to what it is we’re trying to approximate, so we’re now hoping that we get lucky again, and something will “emerge” somehow. Hope, however, is not a strategy. There’s no guarantee. If Demis believes what he says, he should put up a bet with a nontrivial amount. He won’t do that - he’s not stupid.
barfbagginus 3y ago
Yann is respected in the field, and he has some good ideas in that paper and others. But he not advancing the state of the art on LLMs. Ie, he talked about a world model and actor/critic model, but it's other people who are exploring LLM driven world/actor/critic models. If you're not reading papers published in the last months, you're not going to see how quickly things are improving, compared to a year ago. Here's just one example: the incredible explosion in context length. A year ago GPT4 would hallucinate critical details from a 10 page 5000 word essay. Today, Gemini 1.5 can read a 1000 page 5 million word textbook, and recall facts with 99.5% accuracy. This is already an AGI level feat. Now it just needs adaptors, memory modules. AGI doesn't imply it can self improve. It's just generally intelligent and capable of many tasks. Off the shelf LLMs with accessory modules will be enough to build AGI capability and change the nature of work and knowledge.
- ein0p 3y ago
  Yes, we’re now pretty good at associative memory. But associative memory is but a small part of the whole picture. Read the paper, clouds might part a little and enthusiasm might subside. I quite literally work on LLMs as my day job. They alone are not going to somehow get us to AGI
  - barfbagginus 3y ago
    Please tell me which sections of the paper you feel suggest that LLMs + extensions cannot achieve AGI. I have read the paper 3 times in the last year, but I'm not seeing it. There's basically no reason you wouldn't be able to implement LeCunn's entire JEPA architecture using LLM analogues. Associative memory is just one of many huge improvements we could discuss And before you object that LeCunn's methods do online learning with energy methods, let me point to RAG systems that also do online learning - and better than anything Lecunn has published. I own a company where I build new computational geometry codes. And I'm increasingly using LLMs to research, plan, implement, test, and debug my codes. Not only that, but I am an autistic person who increasingly uses LLMs to overcome my social and communication difficulties. GPT is far more empathic and constructive with it's suggestions than any friend, therapist, or advisor I have worked with or have access to. So again it already is AGI for the purpose of helping me process my often complex and baffling emotions. Nobody's saying pure LLMs will be AGI. But pure LLMs working together in novel configurations and using specific components already have that capability. Memory, empathy, planning, agency, theory of mind, etc. You name it, I'll show you a paper where LLMs demonstrate ir. GPT4 by itself does these things better than most people. These capabilities will be packaged into off the shelf components. They will be integrated into post-llm systems that merge these capabilities into a system with the most of the capabilities of a conscious human. That'll be AGI. But what we already have is already AGI as well. You're just not giving it credit.