4 ms·

The AI safety narrative is being pushed by people who think that if we can't control today's AI, we won't be able to control tomorrow's AI either. Yep, current

The AI safety narrative is being pushed by people who think that if we can't control today's AI, we won't be able to control tomorrow's AI either. Yep, current AI harms seem fairly limited. And when gorillas first saw humans on the savanna, the human harms to their habitat were also quite limited. We may only have a single chance to get smarter-than-human AI right.

RandomLensman 4y ago
The AI safety discussions mask that we are nowhere near high powered AGI. Maybe we'll never get there, but the discussions make AI look more powerful than it is, i.e., it's a marketing angle.
- HDThoreaun 4y ago
  If there's a 1% chance of a fast takeoff within the next decade is that not something we should take seriously? I really don't think AGI within the next decade is completely off the table.
  - RandomLensman 4y ago
    What if the chance is only 0.001%? How would we know?
    - HDThoreaun 4y ago
      I mean yea, that's the real question. Not where AI is now, but where it will be a decade or two from now. Talk about AI not currently being dangerous is just ignoring the point AI safety people are making.
      - RandomLensman 4y ago
        Should we worry about other non-existing technology the same way? We had the nano technology people pushing all sorts of scary stuff a few decades ago and nothing came of it, for example.
        HDThoreaun 4y ago
        Maybe. The thing about preparing for unlikely outcomes is that when they don't happen it's easy for detractors to say "see how stupid we were to worry about that" even though everyone agreed that nothing happening was the most likely outcome all along.
        ShredKazoo 4y ago
        Agreed, and the pandemic seems like a good recent example of this.
        RandomLensman 4y ago
        So how worried should we be about the impact of as yet to be invented anti gravity technology, for example? My point is: doomsday scenarios are often used to gain influence/funding/etc., which makes me somewhat skeptical of the motives of people pushing them.
        ShredKazoo 4y ago
        For this argument to be persuasive, you'd have to argue that doomsaying gains more influence/funding/etc. than mainstream AI development. I'm not sure this is true in general? Like, I believe https://intelligence.org/ https://intelligence.org/ has operated on a comparative shoestring for most of its existence, e.g. no salaries above $100K https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-examination https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-ex... The default is: running a nonprofit generates less revenue than running a for-profit. I've talked to a lot of the AI "doomsayers" in person and I believe they are sincere in their worries. That's not to say I agree with all their ideas unreservedly -- there is a fair amount of disagreement among the doomsaying crowd. There is an entire spectrum of beliefs about the likelihood of doomsday, and lots of debate, see e.g. https://www.lesswrong.com/s/n945eovrA3oDueqtq https://www.lesswrong.com/s/n945eovrA3oDueqtq However I don't think this is a Machiavellian ploy for influence/funding/etc.
        pixl97 4y ago
        How many companies are trying to make anti-gravity technology? No serious ones that I know about, we don't even know how that would work. We have 0 examples of anti-gravity. How many companies are working on AI? Well a number of the biggest most well funded companies in the world. And those companies have 8 billion copies of self booting intelligence to learn from. It is incredibly, i mean massively ignorant to say creation of intelligence is impossible when we're doing just that by having sex already. Nature did this by random walk. There was no magic here. No intelligent design. Applied chemistry and randomness created human intelligence. Now humans are taking their massive filtering capabilities of the mind and its reasoning ability it provides and working steadfast on making intelligence happen artificially and your response is about anti-gravity.... Ummm.....
- ShredKazoo 4y ago
  It wasn't even 2 weeks ago that HN users were arguing we are "beyond the point of no return" https://news.ycombinator.com/item?id=34831728 https://news.ycombinator.com/item?id=34831728 and "Pandora's box is open, the genie's bottle is uncorked" https://news.ycombinator.com/item?id=34809607 https://news.ycombinator.com/item?id=34809607 I sit somewhere between the despondent nihilism and the blithe optimism. We owe it to everyone, present and future, to get it right.
ryanblakeley 4y ago
It's easier to not understand the risks. It takes extra effort to understand AI safety problems and more effort again to solve them. People who blow off AI safety are being blithe and not engaging the arguments.
- verteu 4y ago
  As a former Sr ML researcher at Google, I do have a cursory understanding of AI and its risks. But as I mentioned in another comment, the current discussion feels like we're blaming a slot machine's bytecode instead of the casino owners.
  - johnsimer 4y ago
    Right but the eventual discussion is that we need to be able to understand and control the generated bytecode, if possible at all. Eventually smarter than human AIs will exist and we currently do not have real interperability into dumber than human AIs
  - pixl97 4y ago
    Which then you should realize is the crux of the alignment problem. Humans generally have the same alignment constraints, we need food, shelter, air, water, etc. We can all agree on that. But if you give 10 humans $1 and 1 human $1 billion dollars the human with a billion dollars can quickly ignore and forget about the needs of those with $1. The human alignment problem is not solved! With the potential for all knowing information ingestors with a realtime input system that far outstrips human capabilities, the delta of the problem just becomes worse, particularly because the AI goals are much more likely to be aligned with the billionaires ideas.
  - mistrial9 4y ago
    some serious professionals in tech and elsewhere are driven by internal or external forces to do things professionally that may not align well with larger considerations?

RandomLensman 4y ago

The AI safety discussions mask that we are nowhere near high powered AGI. Maybe we'll never get there, but the discussions make AI look more powerful than it is, i.e., it's a marketing angle.

HDThoreaun 4y ago
If there's a 1% chance of a fast takeoff within the next decade is that not something we should take seriously? I really don't think AGI within the next decade is completely off the table.
- RandomLensman 4y ago
  What if the chance is only 0.001%? How would we know?
  - HDThoreaun 4y ago
    I mean yea, that's the real question. Not where AI is now, but where it will be a decade or two from now. Talk about AI not currently being dangerous is just ignoring the point AI safety people are making.
    - RandomLensman 4y ago
      Should we worry about other non-existing technology the same way? We had the nano technology people pushing all sorts of scary stuff a few decades ago and nothing came of it, for example.
      - HDThoreaun 4y ago
        Maybe. The thing about preparing for unlikely outcomes is that when they don't happen it's easy for detractors to say "see how stupid we were to worry about that" even though everyone agreed that nothing happening was the most likely outcome all along.
        ShredKazoo 4y ago
        Agreed, and the pandemic seems like a good recent example of this.
        RandomLensman 4y ago
        So how worried should we be about the impact of as yet to be invented anti gravity technology, for example? My point is: doomsday scenarios are often used to gain influence/funding/etc., which makes me somewhat skeptical of the motives of people pushing them.
        ShredKazoo 4y ago
        For this argument to be persuasive, you'd have to argue that doomsaying gains more influence/funding/etc. than mainstream AI development. I'm not sure this is true in general? Like, I believe https://intelligence.org/ https://intelligence.org/ has operated on a comparative shoestring for most of its existence, e.g. no salaries above $100K https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-examination https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-ex... The default is: running a nonprofit generates less revenue than running a for-profit. I've talked to a lot of the AI "doomsayers" in person and I believe they are sincere in their worries. That's not to say I agree with all their ideas unreservedly -- there is a fair amount of disagreement among the doomsaying crowd. There is an entire spectrum of beliefs about the likelihood of doomsday, and lots of debate, see e.g. https://www.lesswrong.com/s/n945eovrA3oDueqtq https://www.lesswrong.com/s/n945eovrA3oDueqtq However I don't think this is a Machiavellian ploy for influence/funding/etc.
        pixl97 4y ago
        How many companies are trying to make anti-gravity technology? No serious ones that I know about, we don't even know how that would work. We have 0 examples of anti-gravity. How many companies are working on AI? Well a number of the biggest most well funded companies in the world. And those companies have 8 billion copies of self booting intelligence to learn from. It is incredibly, i mean massively ignorant to say creation of intelligence is impossible when we're doing just that by having sex already. Nature did this by random walk. There was no magic here. No intelligent design. Applied chemistry and randomness created human intelligence. Now humans are taking their massive filtering capabilities of the mind and its reasoning ability it provides and working steadfast on making intelligence happen artificially and your response is about anti-gravity.... Ummm.....
ShredKazoo 4y ago
It wasn't even 2 weeks ago that HN users were arguing we are "beyond the point of no return" https://news.ycombinator.com/item?id=34831728 https://news.ycombinator.com/item?id=34831728 and "Pandora's box is open, the genie's bottle is uncorked" https://news.ycombinator.com/item?id=34809607 https://news.ycombinator.com/item?id=34809607 I sit somewhere between the despondent nihilism and the blithe optimism. We owe it to everyone, present and future, to get it right.

ryanblakeley 4y ago

It's easier to not understand the risks. It takes extra effort to understand AI safety problems and more effort again to solve them. People who blow off AI safety are being blithe and not engaging the arguments.

verteu 4y ago
As a former Sr ML researcher at Google, I do have a cursory understanding of AI and its risks. But as I mentioned in another comment, the current discussion feels like we're blaming a slot machine's bytecode instead of the casino owners.
- johnsimer 4y ago
  Right but the eventual discussion is that we need to be able to understand and control the generated bytecode, if possible at all. Eventually smarter than human AIs will exist and we currently do not have real interperability into dumber than human AIs
- pixl97 4y ago
  Which then you should realize is the crux of the alignment problem. Humans generally have the same alignment constraints, we need food, shelter, air, water, etc. We can all agree on that. But if you give 10 humans $1 and 1 human $1 billion dollars the human with a billion dollars can quickly ignore and forget about the needs of those with $1. The human alignment problem is not solved! With the potential for all knowing information ingestors with a realtime input system that far outstrips human capabilities, the delta of the problem just becomes worse, particularly because the AI goals are much more likely to be aligned with the billionaires ideas.
- mistrial9 4y ago
  some serious professionals in tech and elsewhere are driven by internal or external forces to do things professionally that may not align well with larger considerations?