5 ms·

I used to be worried about AI alignment, until I realized something fundamental: We already have unaligned human-level artificial intelligences running around,

I used to be worried about AI alignment, until I realized something fundamental: We already have unaligned human-level artificial intelligences running around, we call them corporations. Now, don't get me wrong, corporations and capitalism in general are doing their best to raze this place, but its really not "The endtimes are upon us", it's more "ugh, I miss Cyberpunk settings being fictional". Heck, even individual humans aren't particularly aligned. In fact, the "AI is going to kill us all" fearmongering is dramatically less alarming than the "What will we do with all the people when we're optional?" question. Which isn't a threat posed by AI, it's a threat posed by people, enabled by AI.

digging 3y ago
> but its really not "The endtimes are upon us" It literally is, though. AI is just the dark horse overtaking our other existential threats in the race to end civilization, but "total ecological collapse" and "nuclear war" are still very strong contenders. Both are driven at least in part (or, almost entirely) by corporate interests. There's also "water shortages" to look out for - make sure to thank Nestle.
- OkayPhysicist 3y ago
  Nuclear war isn't really that big a deal thanks to MAD. No one would be stupid enough to nuke a nuclear armed country, so at worst you end up with a consolidation of countries (a trend we're already seeing, with the EU becoming more and more like a singular sovereignty, expansionism by the Chinese, and the East African Federation). "Total ecological collapse" is overstated: We could make the world a lot shittier to live in before it completely topples civilization. And water shortages are basically the same way: as long as energy is (relatively) plentiful, it's more matter of "making civilization more expensive" than "the endtimes". Again, I'm not saying we shouldn't address these issues. I like a better future rather than a worse one. I'm just saying that we're not sliding into the dark ages anytime soon.
xboxnolifes 3y ago
The worry isn't just that AI wouldn't be aligned, like corporations. The worry is that AI can do what corporations do, but 100x better.
- dragonwriter 3y ago
  No, the worry (such as it is) is that corporations ultimately benefit a class of humans (and at least need other humans to exist to exploit them), whereas AI, if it becomes independent, neither essentially benefits the capitalist class nor essentially needs other classes to exploit. The people most concerned about alignment are capitalists, and they are mostly concerned with the benefit side, since they see aligned AI eliminating at least a large part of the need for the rest of humanity for corporations to provide the benefits it does to them as a plus. While they talk about X-risk, what they try to avoid is that for everyone but themselves, they (especially with the exclusive control of aligned [to their interests] AI that they seek to use fear of unaligned AI to secure) are as much of an X-risk as unaligned AI, and a lot more real and present.
barking_biscuit 3y ago
>I used to be worried about AI alignment, until I realized something fundamental: We already have unaligned human-level artificial intelligences running around, we call them corporations We also call them governments. They can get pretty powerful. >"What will we do with all the people when we're optional?" Judging by the COVID-19 pandemic response, having large aggregates of disempowered individuals from a highly irrational and political species that have become unhappy with the "new normal", it tends to garner some form of reaction. If they are reacting to things that observe and learn from those reactions, and then formulate new goals or sub-goals in response to what they learn, then what is it they might learn and how might they react? The arguments go both ways. The only thing that's clear is that absolutely nobody on earth is going to be able to predict it with any degree of accuracy. You'd have to know too much.
- OkayPhysicist 3y ago
  >> The only thing that's clear is that absolutely nobody on earth is going to be able to predict it with any degree of accuracy. You'd have to know too much. So it is with any technological innovation. Should computers not have been invented because they eliminated jobs? Should steel? What about agriculture? The future is sure to be different, but that doesn't mean we should fight to deny progress. That way lies the Luddite and Conservative. It's only possible to use new tools for good, not try to erase them to prevent evil.
  - barking_biscuit 3y ago
    >So it is with any technological innovation. Should computers not have been invented because they eliminated jobs? Should steel? What about agriculture? The future is sure to be different, but that doesn't mean we should fight to deny progress. That way lies the Luddite and Conservative. It's only possible to use new tools for good, not try to erase them to prevent evil. It's not even necessarily evil, right? It's more likely just one big "whoops, majority of us didn't see that coming, and/or didn't agree with those who said it was". But that whoops lands you in a position that you may not be able to recover from. Ben Geortzal makes your exact argument here: https://youtu.be/MVWzwIg4Adw?t=2795 https://youtu.be/MVWzwIg4Adw?t=2795 and in the same interview he is talking about how his company already spent time and money on building decentralized infrastructure for it to run on "so that no one person can turn it off and stop the singularity" and gives his take that the takeoff is likely to be on the order of years and that it's going to be very rough. So, he at least seems to acknowledge that it's very risky, and that we have no idea what we're actually walking into, but he also just doesn't seem to care if that happens to be a one-way door we walk through and find a big "oopsie" on the other side of. This is the guy credited with popularizing the term AGI in the first place. I dunno man... am I wrong to be worried?
lumenwrites 3y ago
Corporations are WAY more aligned than an AI could be, and people still complain about them. A corporation might pollute the water, kill the dolphins, or give you cancer, but at least it fundamentally doesn't want all humans to be dead. An AI can (and is likely to) have goals that are fundamentally incompatible with the existence of humanity. And, an AI can be way more intelligent and powerful than corporations, so corporations are limited in what they can accomplish when pursuing their interests, but AI might not be.
- gmerc 3y ago
  All the corporation needs is AI to replace its decision makers
dist-epoch 3y ago
The worry is not human-level unaligned AI, but superhuman-level one. There is no superhuman-level corporation yet.
- two_in_one 3y ago
  there is another question moralists are avoiding. Is it moral to keep intelligent creature as a slave? How about forever? So far many realize it may be not safe.
  - OkayPhysicist 3y ago
    There's a huge leap between intelligent and sentient, and approximately 100% of human labor could be satisfied somewhere in between.
- OkayPhysicist 3y ago
  I'd argue that organizations (whether corporation, government, or otherwise) are by definition superhuman. What economic incentive would there be to form companies if an individual could do it all better? Clearly, people working in concert are more effective than those working alone.
  - dist-epoch 3y ago
    By this definition 100 humans pushing a stone have superhuman abilities. This is not what superhuman means in an AI alignment context.
    - digging 3y ago
      > By this definition 100 humans pushing a stone have superhuman abilities. Yes, that is accurate. A person with the strength of 100 people would be superhuman. A group of more-than-one humans has more-than-one-human abilities. > This is not what superhuman means in an AI alignment context. It is though, at least partially. Chess AI are great at chess because they play a shitload of it. Part of the power of AI is just being able to do the same things we do at vastly increased scale. Even if it could only do that, it would be dangerous.
green_man_lives 3y ago
Corporations depend on consumers for their existence. The value they are trying to maximize depends on a functioning supply chain and global economy. So corporations can't outright kill all humans. That symbiotic relationship and constraints aren't really present in an AI the way they are described.
- gmerc 3y ago
  They don’t when robots spend the money.
tipsytoad 3y ago
Why Not Just: Think of AGI Like a Corporation? https://youtu.be/L5pUA3LsEaw https://youtu.be/L5pUA3LsEaw

digging 3y ago

> but its really not "The endtimes are upon us" It literally is, though. AI is just the dark horse overtaking our other existential threats in the race to end civilization, but "total ecological collapse" and "nuclear war" are still very strong contenders. Both are driven at least in part (or, almost entirely) by corporate interests. There's also "water shortages" to look out for - make sure to thank Nestle.

OkayPhysicist 3y ago
Nuclear war isn't really that big a deal thanks to MAD. No one would be stupid enough to nuke a nuclear armed country, so at worst you end up with a consolidation of countries (a trend we're already seeing, with the EU becoming more and more like a singular sovereignty, expansionism by the Chinese, and the East African Federation). "Total ecological collapse" is overstated: We could make the world a lot shittier to live in before it completely topples civilization. And water shortages are basically the same way: as long as energy is (relatively) plentiful, it's more matter of "making civilization more expensive" than "the endtimes". Again, I'm not saying we shouldn't address these issues. I like a better future rather than a worse one. I'm just saying that we're not sliding into the dark ages anytime soon.

xboxnolifes 3y ago

The worry isn't just that AI wouldn't be aligned, like corporations. The worry is that AI can do what corporations do, but 100x better.

dragonwriter 3y ago
No, the worry (such as it is) is that corporations ultimately benefit a class of humans (and at least need other humans to exist to exploit them), whereas AI, if it becomes independent, neither essentially benefits the capitalist class nor essentially needs other classes to exploit. The people most concerned about alignment are capitalists, and they are mostly concerned with the benefit side, since they see aligned AI eliminating at least a large part of the need for the rest of humanity for corporations to provide the benefits it does to them as a plus. While they talk about X-risk, what they try to avoid is that for everyone but themselves, they (especially with the exclusive control of aligned [to their interests] AI that they seek to use fear of unaligned AI to secure) are as much of an X-risk as unaligned AI, and a lot more real and present.

barking_biscuit 3y ago

>I used to be worried about AI alignment, until I realized something fundamental: We already have unaligned human-level artificial intelligences running around, we call them corporations We also call them governments. They can get pretty powerful. >"What will we do with all the people when we're optional?" Judging by the COVID-19 pandemic response, having large aggregates of disempowered individuals from a highly irrational and political species that have become unhappy with the "new normal", it tends to garner some form of reaction. If they are reacting to things that observe and learn from those reactions, and then formulate new goals or sub-goals in response to what they learn, then what is it they might learn and how might they react? The arguments go both ways. The only thing that's clear is that absolutely nobody on earth is going to be able to predict it with any degree of accuracy. You'd have to know too much.

OkayPhysicist 3y ago
>> The only thing that's clear is that absolutely nobody on earth is going to be able to predict it with any degree of accuracy. You'd have to know too much. So it is with any technological innovation. Should computers not have been invented because they eliminated jobs? Should steel? What about agriculture? The future is sure to be different, but that doesn't mean we should fight to deny progress. That way lies the Luddite and Conservative. It's only possible to use new tools for good, not try to erase them to prevent evil.
- barking_biscuit 3y ago
  >So it is with any technological innovation. Should computers not have been invented because they eliminated jobs? Should steel? What about agriculture? The future is sure to be different, but that doesn't mean we should fight to deny progress. That way lies the Luddite and Conservative. It's only possible to use new tools for good, not try to erase them to prevent evil. It's not even necessarily evil, right? It's more likely just one big "whoops, majority of us didn't see that coming, and/or didn't agree with those who said it was". But that whoops lands you in a position that you may not be able to recover from. Ben Geortzal makes your exact argument here: https://youtu.be/MVWzwIg4Adw?t=2795 https://youtu.be/MVWzwIg4Adw?t=2795 and in the same interview he is talking about how his company already spent time and money on building decentralized infrastructure for it to run on "so that no one person can turn it off and stop the singularity" and gives his take that the takeoff is likely to be on the order of years and that it's going to be very rough. So, he at least seems to acknowledge that it's very risky, and that we have no idea what we're actually walking into, but he also just doesn't seem to care if that happens to be a one-way door we walk through and find a big "oopsie" on the other side of. This is the guy credited with popularizing the term AGI in the first place. I dunno man... am I wrong to be worried?

lumenwrites 3y ago

Corporations are WAY more aligned than an AI could be, and people still complain about them. A corporation might pollute the water, kill the dolphins, or give you cancer, but at least it fundamentally doesn't want all humans to be dead. An AI can (and is likely to) have goals that are fundamentally incompatible with the existence of humanity. And, an AI can be way more intelligent and powerful than corporations, so corporations are limited in what they can accomplish when pursuing their interests, but AI might not be.

gmerc 3y ago
All the corporation needs is AI to replace its decision makers

dist-epoch 3y ago

The worry is not human-level unaligned AI, but superhuman-level one. There is no superhuman-level corporation yet.

two_in_one 3y ago
there is another question moralists are avoiding. Is it moral to keep intelligent creature as a slave? How about forever? So far many realize it may be not safe.
- OkayPhysicist 3y ago
  There's a huge leap between intelligent and sentient, and approximately 100% of human labor could be satisfied somewhere in between.
OkayPhysicist 3y ago
I'd argue that organizations (whether corporation, government, or otherwise) are by definition superhuman. What economic incentive would there be to form companies if an individual could do it all better? Clearly, people working in concert are more effective than those working alone.
- dist-epoch 3y ago
  By this definition 100 humans pushing a stone have superhuman abilities. This is not what superhuman means in an AI alignment context.
  - digging 3y ago
    > By this definition 100 humans pushing a stone have superhuman abilities. Yes, that is accurate. A person with the strength of 100 people would be superhuman. A group of more-than-one humans has more-than-one-human abilities. > This is not what superhuman means in an AI alignment context. It is though, at least partially. Chess AI are great at chess because they play a shitload of it. Part of the power of AI is just being able to do the same things we do at vastly increased scale. Even if it could only do that, it would be dangerous.

green_man_lives 3y ago

Corporations depend on consumers for their existence. The value they are trying to maximize depends on a functioning supply chain and global economy. So corporations can't outright kill all humans. That symbiotic relationship and constraints aren't really present in an AI the way they are described.

gmerc 3y ago
They don’t when robots spend the money.

tipsytoad 3y ago

Why Not Just: Think of AGI Like a Corporation? https://youtu.be/L5pUA3LsEaw https://youtu.be/L5pUA3LsEaw