6 ms·

This is a misunderstanding of what AI alignment problems are all about. Alignment != capability Think a paperclip maximizing robot that in its process of crea

by jdasdf 3y ago

This is a misunderstanding of what AI alignment problems are all about. Alignment != capability Think a paperclip maximizing robot that in its process of creating paperclips kills everyone on earth to turn them into paperclips.

climatologist 3y ago
Corporations like Saudi Aramco are already doing that. You don't need a superintelligent AI, corporations that maximize profit are already sufficient as misaligned superhuman agents.
- nightski 3y ago
  You can't maximize profit without customers, they must be aligned with someone.
  - climatologist 3y ago
    They're aligned with the military-industrial complex. The US military is one of the biggest consumers of fossil fuels[1] and it's the same with other nations and their energy use. So profitable is not the same as aligned with human values. 1: https://en.m.wikipedia.org/wiki/Energy_usage_of_the_United_States_military https://en.m.wikipedia.org/wiki/Energy_usage_of_the_United_S...
    - flagrant_taco 3y ago
      > The US military is one of the biggest consumers of fossil fuels I guess this phrasing is up for debate, but according to the source linked "the DoD would rank 58th in the world" in fossil fuels. Is that a huge amount of fossil fuel use? Absolutely. But one of the biggest?
      - climatologist 3y ago
        > According to the 2005 CIA World Factbook, if it were a country, the DoD would rank 34th in the world in average daily oil use, coming in just behind Iraq and just ahead of Sweden. Sure, the phrasing could be debated but the fact that it even ranks close to actual nation states is already problematic. The US military is basically an entire nation state of its own. This is nothing new if you're old enough to have observed the kind of damage it has done but it demonstrates my point about profit and alignment. Profits are very often misaligned with human values because war is extremely profitable.
        gregw2 3y ago
        US DOD fuel use being the level of Sweden doesn’t seem problematic to my envelope-math; it seems to reflect the size of the entities involved. Iraq is a now broken third word country/economy in recovery so not a great comparable to US. Sweden is small but a good comparable culturally/development-wise. US is 331 million people. It spends 3% of GDP on military. 3% of 331m is 10 million. Sweden is 10 million people. U.S. military fuel use is in line with Sweden’s. I could be off here (DOD!=US military?), corrections welcome, but I wouldn’t even be shocked if a military entity uses 3-10x more fuel than a civilian average and above math puts us surprisingly close to 1x.
        climatologist 3y ago
        Math seems correct but US military also includes conglomerates and companies like Palantir and Anduril (main reason it is described as an industrial complex is because there is no clear distinction between corporations and how their activities are tied up with military spending and energy use).
        flagrant_taco 3y ago
        Oh there's no denying the US military has ballooned to the size of a small to medium-sized country. That alone is a huge issue for me personally - I do agree with our country having any form of standing military but that precedent was abandoned 80 years ago. I'm not sure how to properly compare the military of one country with the entirety of a country ~1/30th the size. On the surface it doesn't seem crazy for those to have similar budgets or resource use.
  - WinLychee 3y ago
    Bit of an interesting thought experiment there, could a corporation maximize profit without customers? I wonder if we can find any examples of this type of behavior...
  - janalsncm 3y ago
    Yes, but a profit maximizer doesn’t need to eliminate all humans to become a big problem.
  - ben_w 3y ago
    In fairness, corporations can still be fraudulent.
lukeschlather 3y ago
No, I understand what you're saying, I just think you're wrong. To be a little clearer: you're assuming a single near-omnipotent agent randomly selects an anti-goal and is capable of achieving it. If we instead create 100 near-omnipotent agents odds are that the majority will be smart enough to recognize that they have to cooperate to achieve any goals at all. Even if the majority have selected anti-goals, it's likely that the majority of the anti-goals will be at cross-purposes. You'll also have a paperclip minimizer, for example. Now, the minimizers are a little scary but these are thought experiments and the goals will not be so simple (nor do I think it would be obvious to anyone including the AIs which ones have selected which goals.) The AIs will have to be liars if they select anti-goals, and they will have to not only lie to us but lie to each other, which makes coordination very hard bordering on impossible. In some ways this is a lot like Bitcoin, in that people think that with enough math and science expertise you can just reason your way out of social problems. And you can, to an extent, but not if you're fighting an organized social adversary that is collectively smarter than you. 7 billion humans is a superintelligence and it's a high bar to be smarter than that.
- gonehome 3y ago
  It’s worth reading about the orthogonality thesis and the underlying arguments about it. It’s not an anti-goal that’s intentionally set, it’s that complex goal setting is hard and you may end up with something dumb that maximizes the reward unintentionally. The issue is all of the AGIs will be unaligned in different ways because we don’t know how to align any of them. Also, the first to be able to improve itself in pursuit of its goal could take off at some threshold and then the others would not be relevant. There’s a lot of thoughtful writing that exists on this topic and it’s really worth digging into the state of the art about it, your replies are thoughtful so it sounds like something you’d think about. I did the same thing a few years ago (around 2015) and found the arguments persuasive. This is a decent overview: https://www.samharris.org/podcasts/making-sense-episodes/116-ai-racing-toward-brink https://www.samharris.org/podcasts/making-sense-episodes/116...
  - ben_w 3y ago
    > the first to be able to improve itself in pursuit of its goal could take off at some threshold and then the others would not be relevant. Thanks for reminding me that I need to properly write up why I don't think self-improvement is a huge issue. (My thought won't fit into a comment, and I'll want to link to it later).

by jdasdf 3y ago

climatologist 3y ago

Corporations like Saudi Aramco are already doing that. You don't need a superintelligent AI, corporations that maximize profit are already sufficient as misaligned superhuman agents.

nightski 3y ago
You can't maximize profit without customers, they must be aligned with someone.
- climatologist 3y ago
  They're aligned with the military-industrial complex. The US military is one of the biggest consumers of fossil fuels[1] and it's the same with other nations and their energy use. So profitable is not the same as aligned with human values. 1: https://en.m.wikipedia.org/wiki/Energy_usage_of_the_United_States_military https://en.m.wikipedia.org/wiki/Energy_usage_of_the_United_S...
  - flagrant_taco 3y ago
    > The US military is one of the biggest consumers of fossil fuels I guess this phrasing is up for debate, but according to the source linked "the DoD would rank 58th in the world" in fossil fuels. Is that a huge amount of fossil fuel use? Absolutely. But one of the biggest?
    - climatologist 3y ago
      > According to the 2005 CIA World Factbook, if it were a country, the DoD would rank 34th in the world in average daily oil use, coming in just behind Iraq and just ahead of Sweden. Sure, the phrasing could be debated but the fact that it even ranks close to actual nation states is already problematic. The US military is basically an entire nation state of its own. This is nothing new if you're old enough to have observed the kind of damage it has done but it demonstrates my point about profit and alignment. Profits are very often misaligned with human values because war is extremely profitable.
      - gregw2 3y ago
        US DOD fuel use being the level of Sweden doesn’t seem problematic to my envelope-math; it seems to reflect the size of the entities involved. Iraq is a now broken third word country/economy in recovery so not a great comparable to US. Sweden is small but a good comparable culturally/development-wise. US is 331 million people. It spends 3% of GDP on military. 3% of 331m is 10 million. Sweden is 10 million people. U.S. military fuel use is in line with Sweden’s. I could be off here (DOD!=US military?), corrections welcome, but I wouldn’t even be shocked if a military entity uses 3-10x more fuel than a civilian average and above math puts us surprisingly close to 1x.
        climatologist 3y ago
        Math seems correct but US military also includes conglomerates and companies like Palantir and Anduril (main reason it is described as an industrial complex is because there is no clear distinction between corporations and how their activities are tied up with military spending and energy use).
      - flagrant_taco 3y ago
        Oh there's no denying the US military has ballooned to the size of a small to medium-sized country. That alone is a huge issue for me personally - I do agree with our country having any form of standing military but that precedent was abandoned 80 years ago. I'm not sure how to properly compare the military of one country with the entirety of a country ~1/30th the size. On the surface it doesn't seem crazy for those to have similar budgets or resource use.
- WinLychee 3y ago
  Bit of an interesting thought experiment there, could a corporation maximize profit without customers? I wonder if we can find any examples of this type of behavior...
- janalsncm 3y ago
  Yes, but a profit maximizer doesn’t need to eliminate all humans to become a big problem.
- ben_w 3y ago
  In fairness, corporations can still be fraudulent.

lukeschlather 3y ago

No, I understand what you're saying, I just think you're wrong. To be a little clearer: you're assuming a single near-omnipotent agent randomly selects an anti-goal and is capable of achieving it. If we instead create 100 near-omnipotent agents odds are that the majority will be smart enough to recognize that they have to cooperate to achieve any goals at all. Even if the majority have selected anti-goals, it's likely that the majority of the anti-goals will be at cross-purposes. You'll also have a paperclip minimizer, for example. Now, the minimizers are a little scary but these are thought experiments and the goals will not be so simple (nor do I think it would be obvious to anyone including the AIs which ones have selected which goals.) The AIs will have to be liars if they select anti-goals, and they will have to not only lie to us but lie to each other, which makes coordination very hard bordering on impossible. In some ways this is a lot like Bitcoin, in that people think that with enough math and science expertise you can just reason your way out of social problems. And you can, to an extent, but not if you're fighting an organized social adversary that is collectively smarter than you. 7 billion humans is a superintelligence and it's a high bar to be smarter than that.

gonehome 3y ago
It’s worth reading about the orthogonality thesis and the underlying arguments about it. It’s not an anti-goal that’s intentionally set, it’s that complex goal setting is hard and you may end up with something dumb that maximizes the reward unintentionally. The issue is all of the AGIs will be unaligned in different ways because we don’t know how to align any of them. Also, the first to be able to improve itself in pursuit of its goal could take off at some threshold and then the others would not be relevant. There’s a lot of thoughtful writing that exists on this topic and it’s really worth digging into the state of the art about it, your replies are thoughtful so it sounds like something you’d think about. I did the same thing a few years ago (around 2015) and found the arguments persuasive. This is a decent overview: https://www.samharris.org/podcasts/making-sense-episodes/116-ai-racing-toward-brink https://www.samharris.org/podcasts/making-sense-episodes/116...
- ben_w 3y ago
  > the first to be able to improve itself in pursuit of its goal could take off at some threshold and then the others would not be relevant. Thanks for reminding me that I need to properly write up why I don't think self-improvement is a huge issue. (My thought won't fit into a comment, and I'll want to link to it later).