4 ms·

The AI safety discussions mask that we are nowhere near high powered AGI. Maybe we'll never get there, but the discussions make AI look more powerful than it is

The AI safety discussions mask that we are nowhere near high powered AGI. Maybe we'll never get there, but the discussions make AI look more powerful than it is, i.e., it's a marketing angle.

HDThoreaun 4y ago
If there's a 1% chance of a fast takeoff within the next decade is that not something we should take seriously? I really don't think AGI within the next decade is completely off the table.
- RandomLensman 4y ago
  What if the chance is only 0.001%? How would we know?
  - HDThoreaun 4y ago
    I mean yea, that's the real question. Not where AI is now, but where it will be a decade or two from now. Talk about AI not currently being dangerous is just ignoring the point AI safety people are making.
    - RandomLensman 4y ago
      Should we worry about other non-existing technology the same way? We had the nano technology people pushing all sorts of scary stuff a few decades ago and nothing came of it, for example.
      - HDThoreaun 4y ago
        Maybe. The thing about preparing for unlikely outcomes is that when they don't happen it's easy for detractors to say "see how stupid we were to worry about that" even though everyone agreed that nothing happening was the most likely outcome all along.
        ShredKazoo 4y ago
        Agreed, and the pandemic seems like a good recent example of this.
        RandomLensman 4y ago
        So how worried should we be about the impact of as yet to be invented anti gravity technology, for example? My point is: doomsday scenarios are often used to gain influence/funding/etc., which makes me somewhat skeptical of the motives of people pushing them.
        ShredKazoo 4y ago
        For this argument to be persuasive, you'd have to argue that doomsaying gains more influence/funding/etc. than mainstream AI development. I'm not sure this is true in general? Like, I believe https://intelligence.org/ https://intelligence.org/ has operated on a comparative shoestring for most of its existence, e.g. no salaries above $100K https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-examination https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-ex... The default is: running a nonprofit generates less revenue than running a for-profit. I've talked to a lot of the AI "doomsayers" in person and I believe they are sincere in their worries. That's not to say I agree with all their ideas unreservedly -- there is a fair amount of disagreement among the doomsaying crowd. There is an entire spectrum of beliefs about the likelihood of doomsday, and lots of debate, see e.g. https://www.lesswrong.com/s/n945eovrA3oDueqtq https://www.lesswrong.com/s/n945eovrA3oDueqtq However I don't think this is a Machiavellian ploy for influence/funding/etc.
        pixl97 4y ago
        How many companies are trying to make anti-gravity technology? No serious ones that I know about, we don't even know how that would work. We have 0 examples of anti-gravity. How many companies are working on AI? Well a number of the biggest most well funded companies in the world. And those companies have 8 billion copies of self booting intelligence to learn from. It is incredibly, i mean massively ignorant to say creation of intelligence is impossible when we're doing just that by having sex already. Nature did this by random walk. There was no magic here. No intelligent design. Applied chemistry and randomness created human intelligence. Now humans are taking their massive filtering capabilities of the mind and its reasoning ability it provides and working steadfast on making intelligence happen artificially and your response is about anti-gravity.... Ummm.....
ShredKazoo 4y ago
It wasn't even 2 weeks ago that HN users were arguing we are "beyond the point of no return" https://news.ycombinator.com/item?id=34831728 https://news.ycombinator.com/item?id=34831728 and "Pandora's box is open, the genie's bottle is uncorked" https://news.ycombinator.com/item?id=34809607 https://news.ycombinator.com/item?id=34809607 I sit somewhere between the despondent nihilism and the blithe optimism. We owe it to everyone, present and future, to get it right.

The AI safety discussions mask that we are nowhere near high powered AGI. Maybe we'll never get there, but the discussions make AI look more powerful than it is, i.e., it's a marketing angle.

HDThoreaun 4y ago

If there's a 1% chance of a fast takeoff within the next decade is that not something we should take seriously? I really don't think AGI within the next decade is completely off the table.

RandomLensman 4y ago
What if the chance is only 0.001%? How would we know?
- HDThoreaun 4y ago
  I mean yea, that's the real question. Not where AI is now, but where it will be a decade or two from now. Talk about AI not currently being dangerous is just ignoring the point AI safety people are making.
  - RandomLensman 4y ago
    Should we worry about other non-existing technology the same way? We had the nano technology people pushing all sorts of scary stuff a few decades ago and nothing came of it, for example.
    - HDThoreaun 4y ago
      Maybe. The thing about preparing for unlikely outcomes is that when they don't happen it's easy for detractors to say "see how stupid we were to worry about that" even though everyone agreed that nothing happening was the most likely outcome all along.
      - ShredKazoo 4y ago
        Agreed, and the pandemic seems like a good recent example of this.
      - RandomLensman 4y ago
        So how worried should we be about the impact of as yet to be invented anti gravity technology, for example? My point is: doomsday scenarios are often used to gain influence/funding/etc., which makes me somewhat skeptical of the motives of people pushing them.
        ShredKazoo 4y ago
        For this argument to be persuasive, you'd have to argue that doomsaying gains more influence/funding/etc. than mainstream AI development. I'm not sure this is true in general? Like, I believe https://intelligence.org/ https://intelligence.org/ has operated on a comparative shoestring for most of its existence, e.g. no salaries above $100K https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-examination https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-ex... The default is: running a nonprofit generates less revenue than running a for-profit. I've talked to a lot of the AI "doomsayers" in person and I believe they are sincere in their worries. That's not to say I agree with all their ideas unreservedly -- there is a fair amount of disagreement among the doomsaying crowd. There is an entire spectrum of beliefs about the likelihood of doomsday, and lots of debate, see e.g. https://www.lesswrong.com/s/n945eovrA3oDueqtq https://www.lesswrong.com/s/n945eovrA3oDueqtq However I don't think this is a Machiavellian ploy for influence/funding/etc.
        pixl97 4y ago
        How many companies are trying to make anti-gravity technology? No serious ones that I know about, we don't even know how that would work. We have 0 examples of anti-gravity. How many companies are working on AI? Well a number of the biggest most well funded companies in the world. And those companies have 8 billion copies of self booting intelligence to learn from. It is incredibly, i mean massively ignorant to say creation of intelligence is impossible when we're doing just that by having sex already. Nature did this by random walk. There was no magic here. No intelligent design. Applied chemistry and randomness created human intelligence. Now humans are taking their massive filtering capabilities of the mind and its reasoning ability it provides and working steadfast on making intelligence happen artificially and your response is about anti-gravity.... Ummm.....

ShredKazoo 4y ago

It wasn't even 2 weeks ago that HN users were arguing we are "beyond the point of no return" https://news.ycombinator.com/item?id=34831728 https://news.ycombinator.com/item?id=34831728 and "Pandora's box is open, the genie's bottle is uncorked" https://news.ycombinator.com/item?id=34809607 https://news.ycombinator.com/item?id=34809607 I sit somewhere between the despondent nihilism and the blithe optimism. We owe it to everyone, present and future, to get it right.