6 ms·

It's not - in my experience reading HN - that AGI-sceptics (like me) don't think it will happen, just that we have developed a healthy contempt for the hype-mer

It's not - in my experience reading HN - that AGI-sceptics (like me) don't think it will happen, just that we have developed a healthy contempt for the hype-merchants and their wide-eyed followers who despite years of "machine learning this" and "deep learning that" and "gpt-3 the other" have nothing to show for it in terms of something resembling human-level intelligence, or even any sign of understanding how we think. It's perfectly reasonable to see that and still be confident AGI will happen - just not with current models.

colechristensen 5y ago
I think basically it requires machines that work in fundamentally different ways that can't be reduced to statistics, and that we'll get better at approximating intelligence for a while until we hit a wall where no amount of scale makes any big difference.
ketzo 5y ago
That is a reasonable skepticism, I think. But I will say: by the time anyone has “something to show for it,” AGI will, pretty much by definition, already be here. I don’t think we’re gonna be able to see it coming, or know what it will look like, except through very speculative predictions. So I do think while skepticism is warranted, we should still evaluate everything in good faith, and not automatically jump to shilling. (Not saying you’re doing that, btw - just commenting)
- coderenegade 5y ago
  I think the flight analogy for AI is a good one; we have machines that let us fly, but we don't fly the way birds do. In many ways, the ways that we fly far exceed the capabilities of birds, but there are ways in which birds exceed the capability of our machines. The issue with AGI as it's often framed is that it implicitly assumes a general intelligence like us, without any metric that can relate human intelligence and machine intelligence. We have machines that beat us in Go, so if we took those machines and somehow combined them with machines that beat us in Chess, Starcraft and Dota, are we any closer to AGI than we were previously? What about if that same AI can also drive a car? What's the standard, and where does it end? Where do we fall on that same metric? There are also some fairly deep philosophical questions in this realm, too. For example, to what extent is the human experience (and intelligence) linked to our physical bodies? Cooking by smell and writing by sound are things we do on a daily basis that have no machine analogue. How important is the embodiment principle to AI? Certainly we don't do things by plugging our minds in directly -- we drive cars using the same limbs that we use for everything else, which probably has enormous efficiency advantages. To what extent can you separate the intelligence from the body and the environment? Frankly, I believe we're a lot further away than we think. The last decade has taught us that relatively simple methods can be applied in surprisingly powerful ways, which is an important start, but it doesn't tell us anything about how close we actually are to a given goal, and how we might go about reaching it.
  - soco 5y ago
    The question is: we are further away from what? The goalposts differ wildly, depending on who you ask. From a 1:1 copy of the human mind we are very far, because we also don't have an idea what a human mind is, heck we don't even have a common definition or understanding. Is a soul part of the mind? Should the AI have a soul too? And the rabbit hole goes only further down. So while for some the singularity might be around the corner, for others it's lightyears away. I guess the only way to have everybody agree is if the singularity hits us Skynet-style and I'm definitely not looking forward to that.
    - tomp 5y ago
      > we are further away from what? The goalposts differ wildly, depending on who you ask literally anything the intelligence of a wasp; the intelligence of a cat; the intelligence of a Downs syndrome person; the intelligence of a 3 month old baby AI is very far from all of the above
celdon25 5y ago
Interestingly enough, the man who coined the term "AGI" (Ben Goertzel) has always shared that same skepticism about ML, Deep Learning, and other forms of "Narrow AI".
- ilaksh 5y ago
  Well sure but also now they are working on integrating neural networks and symbolic learning.
eli_gottlieb 5y ago
>have nothing to show for it in terms of something resembling human-level intelligence, or even any sign of understanding how we think. Well of course they have no idea how we think. They're making no actual effort to study how we think. That's for cognitive scientists and neuroscientists, and AI people often sort of dismiss it as having too long a time-horizon or overly high compute requirements before it's applicable to "real world" problems.
- eru 5y ago
  'AI people' is a fairly broad term, and the field is so hot these days that there are plenty of people trying all kinds of different approaches. So there's plenty of scientists and engineers who are trying to understand how the brain works and how to combine that with eg deep learning.
  - eli_gottlieb 5y ago
    On the one hand, sure, I'm one of those scientists. On the other hand, we're still considered a subfield whose work isn't always considered entirely relevant to the rest of the field, because we're measured by the yardstick of task-specific performance on benchmark datasets. I just reviewed a paper yesterday that was powerful and general, but wasn't really trying to hit SOTA on any one test task its architecture could perform. It was trying for generality. This likely got it dinged by the other reviewers.
    - eru 5y ago
      > On the other hand, we're still considered a subfield whose work isn't always considered entirely relevant to the rest of the field, because we're measured by the yardstick of task-specific performance on benchmark datasets. That attitude seems entirely reasonable to me. > I just reviewed a paper yesterday that was powerful and general, but wasn't really trying to hit SOTA on any one test task its architecture could perform. It was trying for generality. This likely got it dinged by the other reviewers. That seems like an interesting paper!
eek2121 5y ago
Hard disagree. I have no contempt, however, as a person of science, I see that we are in the technological equivalent of the stone age when it comes to AI. "Cavemen" probably had some outlandish ideas in their own era as well. To give you an example of how lost we are, we don't even know how human genetics work with the human brain. It is one thing to make a machine that can pretend to think, but to make a machine that not only actually thinks, but also communicates and is able to act in a 'human' or 'intelligent' fashion is exponentially harder. Just my 2 cents. I hope to be proven wrong, but we haven't cured cancer or discovered the secret to fusion yet, so...
- eru 5y ago
  Not sure your analogy holds? Building planes that fly didn't require us to understand how birds or insects fly. Getting directions from Google Maps didn't require them to figure out how hamsters navigate a maze. Now, if you want to build a computer that passes the Turing test, perhaps you need to understand how humans work. Maybe? But it's not clear that this knowledge is necessary to build something smart enough to drown the universe in paperclips. The latter reminds me of Edsger Dijkstra's aphorism: "The question of whether machines can think is about as relevant as the question of whether submarines can swim." (Now, it might turn out that we need to understand how humans tick and how genetics interact with the brain in order to build a successful paperclip optimizer. Probably not, but it might turn out that way. I am just saying that this would be a surprising empirical fact to learn. Not something that we can just assume based on armchair reasoning from analogy.) Now to get slightly off-topic: > I hope to be proven wrong, but we haven't cured cancer or discovered the secret to fusion yet, so... Oh, we can totally build fusion reactors right now! First, a fusor is a bench-top nuclear fusion device. The main downside is that no one has figured out how to get more useful energy out of it then we put in. So probably not what you had in mind. See https://en.wikipedia.org/wiki/Fusor https://en.wikipedia.org/wiki/Fusor Second, we can build a nuclear fusion device that does generate useful energy: You take a huge tank of water, some steam turbines, and a supply of fusion bombs. You take one of the bombs, explode them in the water, and use the turbines to generate electricity. Repeat as needed. It's a very simple system, and we had the means to make this work since the 1950s. Of course, it's also a completely ridiculous design that approximately no-one would want to use in practice. Especially when you already have more conventional nuclear fission reactors. But something along very similar lines was seriously considered for spaceship propulsion. See https://en.wikipedia.org/wiki/Project_Orion_(nuclear_propulsion) https://en.wikipedia.org/wiki/Project_Orion_(nuclear_propuls...
tluyben2 5y ago
> have nothing to show for it in terms of something resembling human-level intelligence We keep moving the goal post; a common theme was when I studied ML in uni in the AI winter of the 90s, that beating Go surely would mean human level intelligence. And many ML models we see now would be considered human level a few decades ago; we moved our goalposts and definitions, which is fine. However ‘resembling’ is vague; I find most stuff on social media (most notably tik tok and instagram) not human level intelligence either, or, reverse, people easily would (and do) believe comments and posts done by something like gpt 3 are done by humans. That is because the level is so low of course. I know gpt 3 is not intelligent (for my definition anyway, which is intentionally vague) but resembling in some cases: definitely.
- benplumley 5y ago
  > beating Go surely would mean human level intelligence With the benefit of hindsight, this just seems like a bad goalpost. Go is hard for computers, but why did anyone think the smarts required to beat a human at it would transfer to understanding or generating text, or anything else a human can do?
- slavik81 5y ago
  > a common theme was when I studied ML in uni in the AI winter of the 90s, that beating Go surely would mean human level intelligence The goalposts haven't moved. If AI researchers thought that general intelligence would be required to beat a human in Go, then they were simply wrong. AlphaGo is not a general-purpose intelligence. It only does one thing. It plays Go.
  - tluyben2 5y ago
    > then they were simply wrong. Sure, point is that every definition we come up with gets beaten and then we were wrong with that definition in the first place; defining what agi means seems hard.
    - robbedpeter 5y ago
      Defining intelligence in a way that captures everything relevant and isn't self referential is extremely difficult. Dr. Marcus Hutter's AIXI is a solid mathematical treatise that reduces intelligence to the concept of information compression, and you can exhaustively construct a logical extrapolation from aixi to any particular feature of intelligence at higher levels, but it's similar to string theory in that it's all-encompassing in scope. It's not useful in narrowing the solution space if you want to build a high level intelligent system. https://en.wikipedia.org/wiki/AIXI https://en.wikipedia.org/wiki/AIXI
simonh 5y ago
I think there are two kinds of AGI skepticism. There's skeptics like us that think we're nowhere even close to beginning to understand how to build an AGI, but see no reason to suppose such a thing is impossible. Then there are those who think AGI is impossible in principle. The people I get most frustrated with are those that use the first argument - we're nowhere close to implementing an AGI - as an argument that therefore AGI is impossible. Sometimes they don't even realise or recognise that this is what they are doing. The argument is often that "we do X and computers don't do X therefore computers can't be like us in that way". Well no computers don't do those things yet because we haven't developed AGI yet. That's not a reasonable argument that developing AGI is impossible.
- FabHK 5y ago
  > Then there are those who think AGI is impossible in principle. Yes. I don't believe that it is impossible in principle either, but it is neither trivial to argue that it is possible. An argument that annoys me to no end is this "well, AI continues to get better (there is no regression in our capability), and therefore it must sooner or later exceed any given threshold, for example human level AGI. QED." That argument is just plain wrong. The world record in the 100-metre dash is only ever improving, but that doesn't mean that humans will soon run Mach 1.
taylodl 5y ago
Human-level intelligence? Dog-level intelligence would be quite the achievement, and we're nowhere close! Will we be there in 20 years? I seriously doubt it - not without a radical departure in processor architecture. We're within striking distance of hitting the limits with our current architecture yet we're nowhere close to being able to achieve human-level intelligence.

colechristensen 5y ago

I think basically it requires machines that work in fundamentally different ways that can't be reduced to statistics, and that we'll get better at approximating intelligence for a while until we hit a wall where no amount of scale makes any big difference.

ketzo 5y ago

That is a reasonable skepticism, I think. But I will say: by the time anyone has “something to show for it,” AGI will, pretty much by definition, already be here. I don’t think we’re gonna be able to see it coming, or know what it will look like, except through very speculative predictions. So I do think while skepticism is warranted, we should still evaluate everything in good faith, and not automatically jump to shilling. (Not saying you’re doing that, btw - just commenting)

coderenegade 5y ago
I think the flight analogy for AI is a good one; we have machines that let us fly, but we don't fly the way birds do. In many ways, the ways that we fly far exceed the capabilities of birds, but there are ways in which birds exceed the capability of our machines. The issue with AGI as it's often framed is that it implicitly assumes a general intelligence like us, without any metric that can relate human intelligence and machine intelligence. We have machines that beat us in Go, so if we took those machines and somehow combined them with machines that beat us in Chess, Starcraft and Dota, are we any closer to AGI than we were previously? What about if that same AI can also drive a car? What's the standard, and where does it end? Where do we fall on that same metric? There are also some fairly deep philosophical questions in this realm, too. For example, to what extent is the human experience (and intelligence) linked to our physical bodies? Cooking by smell and writing by sound are things we do on a daily basis that have no machine analogue. How important is the embodiment principle to AI? Certainly we don't do things by plugging our minds in directly -- we drive cars using the same limbs that we use for everything else, which probably has enormous efficiency advantages. To what extent can you separate the intelligence from the body and the environment? Frankly, I believe we're a lot further away than we think. The last decade has taught us that relatively simple methods can be applied in surprisingly powerful ways, which is an important start, but it doesn't tell us anything about how close we actually are to a given goal, and how we might go about reaching it.
- soco 5y ago
  The question is: we are further away from what? The goalposts differ wildly, depending on who you ask. From a 1:1 copy of the human mind we are very far, because we also don't have an idea what a human mind is, heck we don't even have a common definition or understanding. Is a soul part of the mind? Should the AI have a soul too? And the rabbit hole goes only further down. So while for some the singularity might be around the corner, for others it's lightyears away. I guess the only way to have everybody agree is if the singularity hits us Skynet-style and I'm definitely not looking forward to that.
  - tomp 5y ago
    > we are further away from what? The goalposts differ wildly, depending on who you ask literally anything the intelligence of a wasp; the intelligence of a cat; the intelligence of a Downs syndrome person; the intelligence of a 3 month old baby AI is very far from all of the above

celdon25 5y ago

Interestingly enough, the man who coined the term "AGI" (Ben Goertzel) has always shared that same skepticism about ML, Deep Learning, and other forms of "Narrow AI".

ilaksh 5y ago
Well sure but also now they are working on integrating neural networks and symbolic learning.

eli_gottlieb 5y ago

>have nothing to show for it in terms of something resembling human-level intelligence, or even any sign of understanding how we think. Well of course they have no idea how we think. They're making no actual effort to study how we think. That's for cognitive scientists and neuroscientists, and AI people often sort of dismiss it as having too long a time-horizon or overly high compute requirements before it's applicable to "real world" problems.

eru 5y ago
'AI people' is a fairly broad term, and the field is so hot these days that there are plenty of people trying all kinds of different approaches. So there's plenty of scientists and engineers who are trying to understand how the brain works and how to combine that with eg deep learning.
- eli_gottlieb 5y ago
  On the one hand, sure, I'm one of those scientists. On the other hand, we're still considered a subfield whose work isn't always considered entirely relevant to the rest of the field, because we're measured by the yardstick of task-specific performance on benchmark datasets. I just reviewed a paper yesterday that was powerful and general, but wasn't really trying to hit SOTA on any one test task its architecture could perform. It was trying for generality. This likely got it dinged by the other reviewers.
  - eru 5y ago
    > On the other hand, we're still considered a subfield whose work isn't always considered entirely relevant to the rest of the field, because we're measured by the yardstick of task-specific performance on benchmark datasets. That attitude seems entirely reasonable to me. > I just reviewed a paper yesterday that was powerful and general, but wasn't really trying to hit SOTA on any one test task its architecture could perform. It was trying for generality. This likely got it dinged by the other reviewers. That seems like an interesting paper!

eek2121 5y ago

Hard disagree. I have no contempt, however, as a person of science, I see that we are in the technological equivalent of the stone age when it comes to AI. "Cavemen" probably had some outlandish ideas in their own era as well. To give you an example of how lost we are, we don't even know how human genetics work with the human brain. It is one thing to make a machine that can pretend to think, but to make a machine that not only actually thinks, but also communicates and is able to act in a 'human' or 'intelligent' fashion is exponentially harder. Just my 2 cents. I hope to be proven wrong, but we haven't cured cancer or discovered the secret to fusion yet, so...

eru 5y ago
Not sure your analogy holds? Building planes that fly didn't require us to understand how birds or insects fly. Getting directions from Google Maps didn't require them to figure out how hamsters navigate a maze. Now, if you want to build a computer that passes the Turing test, perhaps you need to understand how humans work. Maybe? But it's not clear that this knowledge is necessary to build something smart enough to drown the universe in paperclips. The latter reminds me of Edsger Dijkstra's aphorism: "The question of whether machines can think is about as relevant as the question of whether submarines can swim." (Now, it might turn out that we need to understand how humans tick and how genetics interact with the brain in order to build a successful paperclip optimizer. Probably not, but it might turn out that way. I am just saying that this would be a surprising empirical fact to learn. Not something that we can just assume based on armchair reasoning from analogy.) Now to get slightly off-topic: > I hope to be proven wrong, but we haven't cured cancer or discovered the secret to fusion yet, so... Oh, we can totally build fusion reactors right now! First, a fusor is a bench-top nuclear fusion device. The main downside is that no one has figured out how to get more useful energy out of it then we put in. So probably not what you had in mind. See https://en.wikipedia.org/wiki/Fusor https://en.wikipedia.org/wiki/Fusor Second, we can build a nuclear fusion device that does generate useful energy: You take a huge tank of water, some steam turbines, and a supply of fusion bombs. You take one of the bombs, explode them in the water, and use the turbines to generate electricity. Repeat as needed. It's a very simple system, and we had the means to make this work since the 1950s. Of course, it's also a completely ridiculous design that approximately no-one would want to use in practice. Especially when you already have more conventional nuclear fission reactors. But something along very similar lines was seriously considered for spaceship propulsion. See https://en.wikipedia.org/wiki/Project_Orion_(nuclear_propulsion) https://en.wikipedia.org/wiki/Project_Orion_(nuclear_propuls...

tluyben2 5y ago

> have nothing to show for it in terms of something resembling human-level intelligence We keep moving the goal post; a common theme was when I studied ML in uni in the AI winter of the 90s, that beating Go surely would mean human level intelligence. And many ML models we see now would be considered human level a few decades ago; we moved our goalposts and definitions, which is fine. However ‘resembling’ is vague; I find most stuff on social media (most notably tik tok and instagram) not human level intelligence either, or, reverse, people easily would (and do) believe comments and posts done by something like gpt 3 are done by humans. That is because the level is so low of course. I know gpt 3 is not intelligent (for my definition anyway, which is intentionally vague) but resembling in some cases: definitely.

benplumley 5y ago
> beating Go surely would mean human level intelligence With the benefit of hindsight, this just seems like a bad goalpost. Go is hard for computers, but why did anyone think the smarts required to beat a human at it would transfer to understanding or generating text, or anything else a human can do?
slavik81 5y ago
> a common theme was when I studied ML in uni in the AI winter of the 90s, that beating Go surely would mean human level intelligence The goalposts haven't moved. If AI researchers thought that general intelligence would be required to beat a human in Go, then they were simply wrong. AlphaGo is not a general-purpose intelligence. It only does one thing. It plays Go.
- tluyben2 5y ago
  > then they were simply wrong. Sure, point is that every definition we come up with gets beaten and then we were wrong with that definition in the first place; defining what agi means seems hard.
  - robbedpeter 5y ago
    Defining intelligence in a way that captures everything relevant and isn't self referential is extremely difficult. Dr. Marcus Hutter's AIXI is a solid mathematical treatise that reduces intelligence to the concept of information compression, and you can exhaustively construct a logical extrapolation from aixi to any particular feature of intelligence at higher levels, but it's similar to string theory in that it's all-encompassing in scope. It's not useful in narrowing the solution space if you want to build a high level intelligent system. https://en.wikipedia.org/wiki/AIXI https://en.wikipedia.org/wiki/AIXI

simonh 5y ago

I think there are two kinds of AGI skepticism. There's skeptics like us that think we're nowhere even close to beginning to understand how to build an AGI, but see no reason to suppose such a thing is impossible. Then there are those who think AGI is impossible in principle. The people I get most frustrated with are those that use the first argument - we're nowhere close to implementing an AGI - as an argument that therefore AGI is impossible. Sometimes they don't even realise or recognise that this is what they are doing. The argument is often that "we do X and computers don't do X therefore computers can't be like us in that way". Well no computers don't do those things yet because we haven't developed AGI yet. That's not a reasonable argument that developing AGI is impossible.

FabHK 5y ago
> Then there are those who think AGI is impossible in principle. Yes. I don't believe that it is impossible in principle either, but it is neither trivial to argue that it is possible. An argument that annoys me to no end is this "well, AI continues to get better (there is no regression in our capability), and therefore it must sooner or later exceed any given threshold, for example human level AGI. QED." That argument is just plain wrong. The world record in the 100-metre dash is only ever improving, but that doesn't mean that humans will soon run Mach 1.

taylodl 5y ago

Human-level intelligence? Dog-level intelligence would be quite the achievement, and we're nowhere close! Will we be there in 20 years? I seriously doubt it - not without a radical departure in processor architecture. We're within striking distance of hitting the limits with our current architecture yet we're nowhere close to being able to achieve human-level intelligence.