3 ms·

There's also the famous Minsky statement that, paraphrasing, AI could pretty much be figured out over a solid summer of work. Appeals to authority on these ques

There's also the famous Minsky statement that, paraphrasing, AI could pretty much be figured out over a solid summer of work. Appeals to authority on these questions are really annoying because people who work their entire lives on a thing are incredibly prone to think it's the most important thing in the world, and that this is the most exceptional time right now. Just statistically that's most of the time wrong. Another example was Hinton, who according to a recent Wired interview, went down his AI doomer spiral after seeing PaLM explain to him why a joke is funny. That is such a bizarre statement it honestly makes me retroactively question some of these people's technical credentials.

version_five 3y ago
A big aspect of the problem is what you might call celebrity worship. People think that because someone is accomplished in an area, we should care what they have to say in another. And AI as in stats and linear algebra has nothing to do with any of the stuff about societal implications or philosophy. So you get a mixed bag of views that basically parallel what laypeople might think, because you're not talking to these people about stuff they're experts in. 1000 years ago an expert in materials and chemicals (say) could build a mirror but still might belive it's a window into the soul or something. The supernatural and practical parts have nothing to do with each other and don't require overlapping expertise.
- pixl97 3y ago
  As a counterpoint, a good number of people worried about AI think about it this way "Think of human intelligence capability merged with machine capability". There is nothing supernatural with that line of thinking. As a human you have some very strict evolutionary limits to the peak of your intelligence. You can't overclock your brain, nor can you easily extend its capabilities outside of the limits of slow analog inputs (eyeballs/ears). It would be the peak of human hubris to think we are the most optimal form of intelligence possible because we are the most optimal form of intelligence that currently exists. I personally believe that far more intelligent architectures than us are possible. What I do not have the answer for is if and when we'll produce them. But if we do, and especially in the short term when the greed of humankind is still limitless, then it will be the death of us.
simonh 3y ago
On the present being an exceptional time, it really is. Technological advances in the last few hundred years have changed the human condition in developed countries to almost unrecognisable levels. I saw this in my grandparents. When I talked to them about what my life was like, much of it made no sense to them. When I showed them an iPad and video called my wife and kids in China, it took a while for it to sink in that this was a live two way video feed to another continent. For quite some time I’m convinced they were worried they were being tricked. I saw them experience real, disorienting future shock multiple times. So in many ways now really is exceptional historically, and this has been the case each generation for the last few generations. It’s likely this will continue to be true. The world my grandchildren grow up in might well be even more exceptional again. As for Hinton, he knows what he is talking about. A.I. alignment is a fiendishly difficult problem. A lot of alignment pathologies that were once theoretical are proving to be real and difficult to avoid phenomena in LLMs. This isn’t vague poorly specified paranoia. There are a lot of very real, verified failure modes for A.I. and for some of them we genuinely have no real idea at all how to properly address them. The reason Hinton changed course is because previously he didn’t think we were anywhere near the point where such problems could pose actual dangers, but the level of advancement in the last few years has been so rapid he’s changed his assessment. I highly recommend Robert Miles A.I. Safety channel on a YouTube. He has a lot of very good introductory videos on many issues in A.I. safety.
storkhm 3y ago
It almost strikes me something else is going on. Some kind of group mental illness. I mean Ray Kurzweil's books were best sellers from what I remember 20 years ago. How could one spend their life working to make things in Kurzweil's books happen and then jump to the idea we are all going to die and it was a mistake when the smallest spark of real AI actually happens? The audience for this is interesting too. It is like no one would want to read the Age of Spiritual Machines today because it is too positive and doesn't have enough doomer in it. Immortality from AI has morphed into extinction from AI as what sells and captures the imagination.

version_five 3y ago

A big aspect of the problem is what you might call celebrity worship. People think that because someone is accomplished in an area, we should care what they have to say in another. And AI as in stats and linear algebra has nothing to do with any of the stuff about societal implications or philosophy. So you get a mixed bag of views that basically parallel what laypeople might think, because you're not talking to these people about stuff they're experts in. 1000 years ago an expert in materials and chemicals (say) could build a mirror but still might belive it's a window into the soul or something. The supernatural and practical parts have nothing to do with each other and don't require overlapping expertise.

pixl97 3y ago
As a counterpoint, a good number of people worried about AI think about it this way "Think of human intelligence capability merged with machine capability". There is nothing supernatural with that line of thinking. As a human you have some very strict evolutionary limits to the peak of your intelligence. You can't overclock your brain, nor can you easily extend its capabilities outside of the limits of slow analog inputs (eyeballs/ears). It would be the peak of human hubris to think we are the most optimal form of intelligence possible because we are the most optimal form of intelligence that currently exists. I personally believe that far more intelligent architectures than us are possible. What I do not have the answer for is if and when we'll produce them. But if we do, and especially in the short term when the greed of humankind is still limitless, then it will be the death of us.

simonh 3y ago

On the present being an exceptional time, it really is. Technological advances in the last few hundred years have changed the human condition in developed countries to almost unrecognisable levels. I saw this in my grandparents. When I talked to them about what my life was like, much of it made no sense to them. When I showed them an iPad and video called my wife and kids in China, it took a while for it to sink in that this was a live two way video feed to another continent. For quite some time I’m convinced they were worried they were being tricked. I saw them experience real, disorienting future shock multiple times. So in many ways now really is exceptional historically, and this has been the case each generation for the last few generations. It’s likely this will continue to be true. The world my grandchildren grow up in might well be even more exceptional again. As for Hinton, he knows what he is talking about. A.I. alignment is a fiendishly difficult problem. A lot of alignment pathologies that were once theoretical are proving to be real and difficult to avoid phenomena in LLMs. This isn’t vague poorly specified paranoia. There are a lot of very real, verified failure modes for A.I. and for some of them we genuinely have no real idea at all how to properly address them. The reason Hinton changed course is because previously he didn’t think we were anywhere near the point where such problems could pose actual dangers, but the level of advancement in the last few years has been so rapid he’s changed his assessment. I highly recommend Robert Miles A.I. Safety channel on a YouTube. He has a lot of very good introductory videos on many issues in A.I. safety.

storkhm 3y ago

It almost strikes me something else is going on. Some kind of group mental illness. I mean Ray Kurzweil's books were best sellers from what I remember 20 years ago. How could one spend their life working to make things in Kurzweil's books happen and then jump to the idea we are all going to die and it was a mistake when the smallest spark of real AI actually happens? The audience for this is interesting too. It is like no one would want to read the Age of Spiritual Machines today because it is too positive and doesn't have enough doomer in it. Immortality from AI has morphed into extinction from AI as what sells and captures the imagination.