19 ms·

> we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential

> we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Who is we exactly? As someone working in AI research I know no one that would agree with this statement, so im quite puzzled by that statement.

Isinlor 4y ago
Nando de Freitas Research Director at @DeepMind. CIFAR. Previously Prof @UBC & @UniofOxford. made a lot of headlines: https://twitter.com/NandoDF/status/1525397036325019649 https://twitter.com/NandoDF/status/1525397036325019649 Someone’s opinion article. My opinion: It’s all about scale now! The Game is Over! It’s about making these models bigger, safer, compute efficient, faster at sampling, smarter memory, more modalities, INNOVATIVE DATA, on/offline, … 1/N Solving these scaling challenges is what will deliver AGI. Research focused on these problems, eg S4 for greater memory, is needed. Philosophy about symbols isn’t. Symbols are tools in the world and big nets have no issue creating them and manipulating them 2/n https://twitter.com/NandoDF/status/1525397036325019649 https://twitter.com/NandoDF/status/1525397036325019649
- sailingparrot 4y ago
  And I agree with Nando’s view, but he is not saying we can just take a transformer model, scale it 10T parameters and get AGI. He is only saying that trying to reach AGI with a « smarter » algorithm is hopeless, what matters is scale, similar to Sutton’a bitter lesson. But we still need to work on getting systems that scale better, that are more compute efficient etc. And no one knows how far we have to scale. So saying AGI will just be « transformer + RL » to me seems ridiculous. Many more breakthroughs are needed.
ml_basics 4y ago
I work in AI and would roughly agree with it to first order. For me the key breakthrough has been seeing how large transformers trained with big datasets have shown incredible performance in completely different data modalities (text, image, and probably soon others too). This was absolutely not expected by most researchers 5 years ago.
- riku_iki 4y ago
  past breakthrough doesn't guarantee future breakthroughs.
  - trention 4y ago
    The old problem of induction. It's interesting how many of the AI accelerationist crowd are (purposefully pretending to be) unaware of it.
    - monktastic1 4y ago
      The problem is, the POI applies to literally everything, including events which seem to have well-understood and well-established causes. That makes it impossible to know when it's going to be applicable in the real world and when it's not.
  - why_only_15 4y ago
    but they're certainly indicative.
    - riku_iki 4y ago
      yes, indicative of past breakthrough )
- godelski 4y ago
  The problem here is that transformers aren't always the best tool. They are powerful because they can capture large receptive fields but ConvNext even shows that a convolution can still out perform Transformers on some tasks. There's plenty of modern CNNs that have both small and large receptive fields that perform better than transformers. But of course there is no one size fits all architecture and it is fairly naive to think so. I think what was more surprising with transformers is that you could scale them well and perform complex tasks with them alone. But I think a big part you're ignoring here is how training has changed in the last 5 years. You look back at something like VGG where we go depthwise with CNNs vs something like Swin and there's a ton more complexity in the latter (and difficulties in reproduction of results likely due to this). There's vastly more uses of data augmentation, different types of normalization (see ConvNext), better learning rate schedules, and massive compute allows for a much better hyper-parameter search (I do also want to note that there are problems in this area as many are HP searching over the test set and not a validation or training set). Yeah, the field has come a long way and I fully expect it to continue to grow like crazy but AGI/HLI is a vastly more complex problem that we have a lot of evidence that the current methods won't get us there.
version_five 4y ago
> Who is we exactly? When I read these kind of threads, I believe it's "enthusiast" laypeople who follow the headlines but don't actually have a deep understanding of the tech. Of course there are the promoters who are raising money and need to frame each advance in the most optimistic light. I don't see anything wrong with that, it just means that there will be a group of techie but not research literate folks who almost necessarily become the promoters and talk about how such and such headline means that a big advance is right around the corner. That is what I believe we're seeing here.
- wruza 4y ago
  Can someone please explain like we are fifteen why AGI is impossible, at least right now? Or if not AGI, then something similar to a cat/etc mind? As far as I am imagining it, current models are pipelines of various trained networks (and more traditional filters in the mix) that operate like request-reply. Why can’t you just connect few different pipelines in a loop/graph and make an autonomous self-feeding entity? By different I mean not looping gpt to itself, but different like object detection from camera vs emotion from a picture based on some training data. Is it because you don’t have data what is scary or not, or for a completely different reason?
  - grayclhn 4y ago
    Cool, make it happen. Don’t know how? Neither does anyone else. That’s why it’s not happening right now.
  - godelski 4y ago
    > Can someone please explain like we are fifteen why AGI is impossible, at least right now Probably not. I did attempt to give a high level explanation in another comment but I think there is this naive belief that complex problems can be distilled into terms that laymen can understand. This is such a complex problem that is so ill-defined that experts argue. I'm not sure there's really a good "explain it like I'm an undergrad who's done ML courses" explanation that can be concisely summed up in a HN comment.
Jack000 4y ago
Which part do you find objectionable - the lack of progress in previous years or the current/future potential of transformers in RL? I do work in ML but mostly applications instead of research.
- sailingparrot 4y ago
  I think there is a lot of potential for transformers in RL. And I do think that AGI is mostly a matter of scale. But I do not agree at all that AGI is just transformers + RL. I don’t think we have a concrete idea at all, and I don’t agree at all that 10 years ago people thought we had no path forward. 10 years ago is exactly AlexNet, aka the birth of deep learning.
  - Jack000 4y ago
    I think it's fair to say that no one knows if AGI is even possible at this point, but RL+transformer is definitely the most promising approach imo. 10 years ago no one was talking about AGI, researchers didn't even call it "AI" due to its association with the previous AI winter. There were certainly advances in ML but I think if you asked anyone whether those approaches would lead to real AI it would be an unequivocal no.
    - dinosaurdynasty 4y ago
      I'd say AGI is very very much possible because it already exists (for example: humans) and unless there exists a soul or some other kind of "literally magic" (and I'd be doubtful even then) there is no reason we couldn't one day build AGI. It also seems like unless something goes very very wrong with the exponential growth curve in computational capacity humanity has had, we'll be able to straight up simulate a human brain (probably well before) 2100.
- Calavar 4y ago
  One big issue is how exactly we'll continue to scale. Exponential growth is hard to maintain. An example: In the Chinchilla paper [1], the authors suggest that most big transformer models are undertrained, and that we will probably see diminishing returns in scaling up the size of networks if we don't also scale up the size of the datasets. They have a subanalysis where they extrapolate out how big datasets will need to be for larger models. If you believe it, then within two to three generations of big NLG models, we may need on the order of 100 trillion text tokens. But it's not clear if that much text even exists. [1] https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556
godelski 4y ago
> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.
- ta988 4y ago
  Yes we are going back to symbolism, , ontologies and causality. Which is a good and healthy sign. It was abandonned the same way neural networks were at some point. Now that it becomes practical at scale, we can combine them all.
- thewarrior 4y ago
  I know it maybe too much to ask but as a noob to this field I’d love to hear more about why AGI is not feasible even in 50 years.
  - version_five 4y ago
    I've seen a few posts here asking why it isn't possible. Research doesn't work like that, you can't just imagine something is possible and demand reasons why it isn't. How is it possible? It feels a bit like a lot of these fringe science things (electromagnetic fields causing harm etc) where people's argument is that nobody's shown it isn't true. Well if it's true, propose a mechanism and study it. You're not really doing research if you just make up a conclusion and use the fact that it can't be definitively refuted as evidence for it
    - godelski 4y ago
      Yeah this is a good point. I've been taking the bearish position of HLI here but that doesn't mean I don't think HLI is impossible. I very much do think it is possible (I'm not sure there are many researchers who think it is impossible) but just how close we are. But we don't even have good definitions, so really this is just laymen being privy to expert arguments. I also think there's this naive belief that complex topics can be distilled to laymen terms while maintaining accuracy. That's definitely not true here, as even our "in-group" doesn't have a consensus. But that's cutting edge research for you.
silverlake 4y ago
I agree with you. However there’s a camp that thinks bigger models will lead to ~AGI (whatever that is). Another believes reinforcement learning will get there somehow. LeCun published an alternative vague idea because he thinks neither will get to AGI.
mkaic 4y ago
Fellow AI researcher here who personal agrees with OP, there are some of us that are believers ;). I agree that most people in our industry are generally not very bullish about near-term AGI, I think it just comes down to how you perceive the complexity of the problem. Personally, I think it's solvable because passing a lengthy, adversarial Turing test reliably is all it would take to convince me of sentience in a model, and well, GPT-3 is already close to that (though I want to clarify that I do not believe any present-day models possess any inkling sentience yet). I think the main three obstacles in our way are imbuing models with (from least to most difficult): 1. continuous execution, which would be necessary for a real-time "stream of consciousness" 2. discreet short and long term memory and the ability to use it productively 3. plasticity at runtime 1 is just a matter of finding more efficient architectures and cheapening compute, both fronts we're constantly making progress on. 2 is a little more muddy, but there are already some really impressive models that have retrieval mechanisms which allow them to search through databases to help them contextualize prompts which I believe is a step in the right direction. 3 is probably the hardest in my opinion--I think we need several fundamental breakthrough discoveries before our models are anywhere near as plastic as, say, a human brain. My hypothesis is that exceptional plasticity is necessary for sentience, but obviously that's just my own personal opinion. Anyways, thought I'd chime in with my two cents. I'd love to hear others' thoughts on the feasibility/infeasibility of near-term AGI.
- mementomorti 4y ago
  I'm not in the field at all but just generally curious regarding #3. I'm under the impression that a lot of plasticity in our brains is related to sleep, are there areas of AI research following that thread, plasticity at downtime so to say?
- Jack000 4y ago
  "Sentience" is a weird word because it's a such fuzzy concept, but I think if we define it properly there are testable answers to the question of machine sentience. I would define sentience as an awareness of oneself in relation to one's environment, that generalizes to different aspects of self and environment (eg. social, physical etc) One of the key aspects of intelligence is learning from 3rd-person perspectives - eg. you observe another ape hunting with a stick, and decide to emulate that behavior for yourself. This seemingly simple behavior requires a lot of work, you have to recognize other apes as agents similar to yourself, then map their actions to your own body, then perform those actions and see if it had the intended outcome. Current RL agents are not capable of this. The data used for GATO and similar agents for imitation learning are in first person perspective. If an agent could learn from a 3rd person perspective, via direct observations of its environment, I would say that would be the beginning of machine sentience.
pjfin123 4y ago
A RL agent with a large Transformer seems to me like a design that would build an AGI.

Isinlor 4y ago

Nando de Freitas Research Director at @DeepMind. CIFAR. Previously Prof @UBC & @UniofOxford. made a lot of headlines: https://twitter.com/NandoDF/status/1525397036325019649 https://twitter.com/NandoDF/status/1525397036325019649 Someone’s opinion article. My opinion: It’s all about scale now! The Game is Over! It’s about making these models bigger, safer, compute efficient, faster at sampling, smarter memory, more modalities, INNOVATIVE DATA, on/offline, … 1/N Solving these scaling challenges is what will deliver AGI. Research focused on these problems, eg S4 for greater memory, is needed. Philosophy about symbols isn’t. Symbols are tools in the world and big nets have no issue creating them and manipulating them 2/n https://twitter.com/NandoDF/status/1525397036325019649 https://twitter.com/NandoDF/status/1525397036325019649

sailingparrot 4y ago
And I agree with Nando’s view, but he is not saying we can just take a transformer model, scale it 10T parameters and get AGI. He is only saying that trying to reach AGI with a « smarter » algorithm is hopeless, what matters is scale, similar to Sutton’a bitter lesson. But we still need to work on getting systems that scale better, that are more compute efficient etc. And no one knows how far we have to scale. So saying AGI will just be « transformer + RL » to me seems ridiculous. Many more breakthroughs are needed.

ml_basics 4y ago

I work in AI and would roughly agree with it to first order. For me the key breakthrough has been seeing how large transformers trained with big datasets have shown incredible performance in completely different data modalities (text, image, and probably soon others too). This was absolutely not expected by most researchers 5 years ago.

riku_iki 4y ago
past breakthrough doesn't guarantee future breakthroughs.
- trention 4y ago
  The old problem of induction. It's interesting how many of the AI accelerationist crowd are (purposefully pretending to be) unaware of it.
  - monktastic1 4y ago
    The problem is, the POI applies to literally everything, including events which seem to have well-understood and well-established causes. That makes it impossible to know when it's going to be applicable in the real world and when it's not.
- why_only_15 4y ago
  but they're certainly indicative.
  - riku_iki 4y ago
    yes, indicative of past breakthrough )
godelski 4y ago
The problem here is that transformers aren't always the best tool. They are powerful because they can capture large receptive fields but ConvNext even shows that a convolution can still out perform Transformers on some tasks. There's plenty of modern CNNs that have both small and large receptive fields that perform better than transformers. But of course there is no one size fits all architecture and it is fairly naive to think so. I think what was more surprising with transformers is that you could scale them well and perform complex tasks with them alone. But I think a big part you're ignoring here is how training has changed in the last 5 years. You look back at something like VGG where we go depthwise with CNNs vs something like Swin and there's a ton more complexity in the latter (and difficulties in reproduction of results likely due to this). There's vastly more uses of data augmentation, different types of normalization (see ConvNext), better learning rate schedules, and massive compute allows for a much better hyper-parameter search (I do also want to note that there are problems in this area as many are HP searching over the test set and not a validation or training set). Yeah, the field has come a long way and I fully expect it to continue to grow like crazy but AGI/HLI is a vastly more complex problem that we have a lot of evidence that the current methods won't get us there.

version_five 4y ago

> Who is we exactly? When I read these kind of threads, I believe it's "enthusiast" laypeople who follow the headlines but don't actually have a deep understanding of the tech. Of course there are the promoters who are raising money and need to frame each advance in the most optimistic light. I don't see anything wrong with that, it just means that there will be a group of techie but not research literate folks who almost necessarily become the promoters and talk about how such and such headline means that a big advance is right around the corner. That is what I believe we're seeing here.

wruza 4y ago
Can someone please explain like we are fifteen why AGI is impossible, at least right now? Or if not AGI, then something similar to a cat/etc mind? As far as I am imagining it, current models are pipelines of various trained networks (and more traditional filters in the mix) that operate like request-reply. Why can’t you just connect few different pipelines in a loop/graph and make an autonomous self-feeding entity? By different I mean not looping gpt to itself, but different like object detection from camera vs emotion from a picture based on some training data. Is it because you don’t have data what is scary or not, or for a completely different reason?
- grayclhn 4y ago
  Cool, make it happen. Don’t know how? Neither does anyone else. That’s why it’s not happening right now.
- godelski 4y ago
  > Can someone please explain like we are fifteen why AGI is impossible, at least right now Probably not. I did attempt to give a high level explanation in another comment but I think there is this naive belief that complex problems can be distilled into terms that laymen can understand. This is such a complex problem that is so ill-defined that experts argue. I'm not sure there's really a good "explain it like I'm an undergrad who's done ML courses" explanation that can be concisely summed up in a HN comment.

Jack000 4y ago

Which part do you find objectionable - the lack of progress in previous years or the current/future potential of transformers in RL? I do work in ML but mostly applications instead of research.

sailingparrot 4y ago
I think there is a lot of potential for transformers in RL. And I do think that AGI is mostly a matter of scale. But I do not agree at all that AGI is just transformers + RL. I don’t think we have a concrete idea at all, and I don’t agree at all that 10 years ago people thought we had no path forward. 10 years ago is exactly AlexNet, aka the birth of deep learning.
- Jack000 4y ago
  I think it's fair to say that no one knows if AGI is even possible at this point, but RL+transformer is definitely the most promising approach imo. 10 years ago no one was talking about AGI, researchers didn't even call it "AI" due to its association with the previous AI winter. There were certainly advances in ML but I think if you asked anyone whether those approaches would lead to real AI it would be an unequivocal no.
  - dinosaurdynasty 4y ago
    I'd say AGI is very very much possible because it already exists (for example: humans) and unless there exists a soul or some other kind of "literally magic" (and I'd be doubtful even then) there is no reason we couldn't one day build AGI. It also seems like unless something goes very very wrong with the exponential growth curve in computational capacity humanity has had, we'll be able to straight up simulate a human brain (probably well before) 2100.
Calavar 4y ago
One big issue is how exactly we'll continue to scale. Exponential growth is hard to maintain. An example: In the Chinchilla paper [1], the authors suggest that most big transformer models are undertrained, and that we will probably see diminishing returns in scaling up the size of networks if we don't also scale up the size of the datasets. They have a subanalysis where they extrapolate out how big datasets will need to be for larger models. If you believe it, then within two to three generations of big NLG models, we may need on the order of 100 trillion text tokens. But it's not clear if that much text even exists. [1] https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556

godelski 4y ago

> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.

ta988 4y ago
Yes we are going back to symbolism, , ontologies and causality. Which is a good and healthy sign. It was abandonned the same way neural networks were at some point. Now that it becomes practical at scale, we can combine them all.
thewarrior 4y ago
I know it maybe too much to ask but as a noob to this field I’d love to hear more about why AGI is not feasible even in 50 years.
- version_five 4y ago
  I've seen a few posts here asking why it isn't possible. Research doesn't work like that, you can't just imagine something is possible and demand reasons why it isn't. How is it possible? It feels a bit like a lot of these fringe science things (electromagnetic fields causing harm etc) where people's argument is that nobody's shown it isn't true. Well if it's true, propose a mechanism and study it. You're not really doing research if you just make up a conclusion and use the fact that it can't be definitively refuted as evidence for it
  - godelski 4y ago
    Yeah this is a good point. I've been taking the bearish position of HLI here but that doesn't mean I don't think HLI is impossible. I very much do think it is possible (I'm not sure there are many researchers who think it is impossible) but just how close we are. But we don't even have good definitions, so really this is just laymen being privy to expert arguments. I also think there's this naive belief that complex topics can be distilled to laymen terms while maintaining accuracy. That's definitely not true here, as even our "in-group" doesn't have a consensus. But that's cutting edge research for you.

silverlake 4y ago

I agree with you. However there’s a camp that thinks bigger models will lead to ~AGI (whatever that is). Another believes reinforcement learning will get there somehow. LeCun published an alternative vague idea because he thinks neither will get to AGI.

mkaic 4y ago

Fellow AI researcher here who personal agrees with OP, there are some of us that are believers ;). I agree that most people in our industry are generally not very bullish about near-term AGI, I think it just comes down to how you perceive the complexity of the problem. Personally, I think it's solvable because passing a lengthy, adversarial Turing test reliably is all it would take to convince me of sentience in a model, and well, GPT-3 is already close to that (though I want to clarify that I do not believe any present-day models possess any inkling sentience yet). I think the main three obstacles in our way are imbuing models with (from least to most difficult): 1. continuous execution, which would be necessary for a real-time "stream of consciousness" 2. discreet short and long term memory and the ability to use it productively 3. plasticity at runtime 1 is just a matter of finding more efficient architectures and cheapening compute, both fronts we're constantly making progress on. 2 is a little more muddy, but there are already some really impressive models that have retrieval mechanisms which allow them to search through databases to help them contextualize prompts which I believe is a step in the right direction. 3 is probably the hardest in my opinion--I think we need several fundamental breakthrough discoveries before our models are anywhere near as plastic as, say, a human brain. My hypothesis is that exceptional plasticity is necessary for sentience, but obviously that's just my own personal opinion. Anyways, thought I'd chime in with my two cents. I'd love to hear others' thoughts on the feasibility/infeasibility of near-term AGI.

mementomorti 4y ago
I'm not in the field at all but just generally curious regarding #3. I'm under the impression that a lot of plasticity in our brains is related to sleep, are there areas of AI research following that thread, plasticity at downtime so to say?
Jack000 4y ago
"Sentience" is a weird word because it's a such fuzzy concept, but I think if we define it properly there are testable answers to the question of machine sentience. I would define sentience as an awareness of oneself in relation to one's environment, that generalizes to different aspects of self and environment (eg. social, physical etc) One of the key aspects of intelligence is learning from 3rd-person perspectives - eg. you observe another ape hunting with a stick, and decide to emulate that behavior for yourself. This seemingly simple behavior requires a lot of work, you have to recognize other apes as agents similar to yourself, then map their actions to your own body, then perform those actions and see if it had the intended outcome. Current RL agents are not capable of this. The data used for GATO and similar agents for imitation learning are in first person perspective. If an agent could learn from a 3rd person perspective, via direct observations of its environment, I would say that would be the beginning of machine sentience.

pjfin123 4y ago

A RL agent with a large Transformer seems to me like a design that would build an AGI.