4 ms·

Until the alignment movement begins to take seriously the idea that we already have misaligned artificial general intelligences I think they are best viewed as

Until the alignment movement begins to take seriously the idea that we already have misaligned artificial general intelligences I think they are best viewed as a convenient foil Paperclip maximizers exist, they're made not only of code but of people

mshron 4y ago
Some of us do! Check out a whitepaper on that exact point: https://ai.objectives.institute/whitepaper https://ai.objectives.institute/whitepaper It’s weird to have been working on a paper for almost a year and have it launch into this environment, but uptake has been good. My hope is that we will continue to see more nuance around different kinds of alignment risks in the near future. There’s a wide spectrum between biased statistical models and paperclip maximizing overlords, and lots bad but not existentially catastrophic things for the public to want to keep a pulse on.
- walleeee 4y ago
  Thanks! Looks like good work. I hope this idea continues to get traction: > In some sense, we’re already living in a world of misaligned optimizers I understand this is an academic paper given to nuance and understatement, but for any drive-by readers, this is true in an extremely literal sense, with very real consequences.
PoignardAzur 4y ago
Reminds me of the parody in Scott Alexander's article "If the media reported on other things like it does EA" > Some epidemiologists are worrying that a new virus from Wuhan could become a wider catastrophe. Their message is infecting people around the world with fear and xenophobia, spreading faster than any plague. Perhaps they should consider that in some sense, they themselves are the global pandemic. Like, yeah, people did consider that idea, and the "corporations are the real unaligned AI" idea, and the "capitalism is the real extinction risk" idea, and all the pseudo-clever variations of the concept. The problem is that "understanding that capitalism has problems" isn't equivalent to "having an actionable plan to solve capitalism".
- walleeee 4y ago
  > The problem is that "understanding that capitalism has problems" isn't equivalent to "having an actionable plan to solve capitalism This is a caricature and the same thing could be said of AI x-risk. There are plenty of ideas on how to avoid unwanted effects of economic systems. I don't think it's at all clear that it's a single problem with a single solution. Getting ideas into practice tends to be the tougher challenge. More broadly, the point is not to say "wow, alignment problems have existed for a long time already!" This is not profound or clever, it's obvious. But there's a big group of people considering a narrow definition of the problem, and playing what could be considered a useful social role.
diputsmonro 4y ago
Precisely! I'm much less concerned about super-intelligent AIs and much more concerned with shortsighted, greedy humans using pretty-good AIs (like those we have now) to squeeze out every ounce of profit from our already misaligned systems, at the expense of everyone else. Not to mention the political implications of being able to convincingly fake voices, photos, and videos. In this sense, I'm pleased to see Open AI claim to be taking a more careful stance, but to be honest I think the genie is already out of the bottle.

mshron 4y ago

Some of us do! Check out a whitepaper on that exact point: https://ai.objectives.institute/whitepaper https://ai.objectives.institute/whitepaper It’s weird to have been working on a paper for almost a year and have it launch into this environment, but uptake has been good. My hope is that we will continue to see more nuance around different kinds of alignment risks in the near future. There’s a wide spectrum between biased statistical models and paperclip maximizing overlords, and lots bad but not existentially catastrophic things for the public to want to keep a pulse on.

walleeee 4y ago
Thanks! Looks like good work. I hope this idea continues to get traction: > In some sense, we’re already living in a world of misaligned optimizers I understand this is an academic paper given to nuance and understatement, but for any drive-by readers, this is true in an extremely literal sense, with very real consequences.

PoignardAzur 4y ago

Reminds me of the parody in Scott Alexander's article "If the media reported on other things like it does EA" > Some epidemiologists are worrying that a new virus from Wuhan could become a wider catastrophe. Their message is infecting people around the world with fear and xenophobia, spreading faster than any plague. Perhaps they should consider that in some sense, they themselves are the global pandemic. Like, yeah, people did consider that idea, and the "corporations are the real unaligned AI" idea, and the "capitalism is the real extinction risk" idea, and all the pseudo-clever variations of the concept. The problem is that "understanding that capitalism has problems" isn't equivalent to "having an actionable plan to solve capitalism".

walleeee 4y ago
> The problem is that "understanding that capitalism has problems" isn't equivalent to "having an actionable plan to solve capitalism This is a caricature and the same thing could be said of AI x-risk. There are plenty of ideas on how to avoid unwanted effects of economic systems. I don't think it's at all clear that it's a single problem with a single solution. Getting ideas into practice tends to be the tougher challenge. More broadly, the point is not to say "wow, alignment problems have existed for a long time already!" This is not profound or clever, it's obvious. But there's a big group of people considering a narrow definition of the problem, and playing what could be considered a useful social role.

diputsmonro 4y ago

Precisely! I'm much less concerned about super-intelligent AIs and much more concerned with shortsighted, greedy humans using pretty-good AIs (like those we have now) to squeeze out every ounce of profit from our already misaligned systems, at the expense of everyone else. Not to mention the political implications of being able to convincingly fake voices, photos, and videos. In this sense, I'm pleased to see Open AI claim to be taking a more careful stance, but to be honest I think the genie is already out of the bottle.