13 ms·

I have yet to see a solution for “AI safety” that doesn’t involve ceding control of our most powerful models to a small handful of corporations. It’s hard to t

by valine 3y ago

I have yet to see a solution for “AI safety” that doesn’t involve ceding control of our most powerful models to a small handful of corporations. It’s hard to take these safety concerns seriously when the organizations blowing the whistle are simultaneously positioning themselves to capture the majority of the value.

circuit10 3y ago
We don’t really have a good solution, I guess that’s why we need more research into it Companies might argue that giving them control might help but I don’t think most individuals working on it think that will work
- sycamoretrees 3y ago
  Is more research really going to offer any true solutions? I’d be genuinely interested in hearing about what research could potentially offer (the development of tools to counter AI disinformation? A deeper understanding of how LLMs work?), but it seems to me that the only “real” solution is ultimately political. The issue is that it would require elements of authoritarianism and censorship.
  - wongarsu 3y ago
    A lot of research about avoiding extinction by AI is about alignment. LLMs are pretty harmless in that they (currently) don't have any goals, they just produce text. But at some point we will succeed in turning them into "thinking" agents that try to achieve a goal. Similar to a chess AI, but interacting with the real world instead. One of the big problems with that is that we don't have a good way to make sure the goals of the AI match what we want it to do. Even if the whole "human governance" political problem were solved, we still couldn't reliably control any AI. Solving that is a whole research field. Building better ways to understand the inner workings of neural networks is definitely one avenue
    - sycamoretrees 3y ago
      I see. Thanks for the reply. But I wonder if that’s not a bit too optimistic and not concrete enough. Alignment won’t solve the world’s woes, just like “enlightenment” (a word which sounds a lot like alignment and which is similarly undefinable) does not magically rectify the realities of the world. Why should bad actors care about alignment? Another example is climate change. We have a lot of good ideas which, combined, would stop us from killing millions of people across the world. We have the research - is more “research” really the key?
    - pixl97 3y ago
      Intelligence cannot be 'solved', I would go on to further say that an intelligence without the option of violence isn't an intelligence at all. If you suddenly wanted to kill people, for example, then could probably kill a few before you were stopped. That is typically the limits of an individuals power. Now, if you were a corporation with money, depending on the strategy you used you could likely kill anywhere from hundreds to hundreds of thousands. Kick it up to government level, and well, the term "just a statistic" exists for a reason. We tend to have laws around these behaviors, but they are typically punitive. The law realizes that humans, and human systems will unalign themselves from "moral" behavior (whatever that may be considered at the time). When the lawgiver itself becomes unaligned, well, things tend to get bad. Human alignment typically consists of benefits (I give you nice things/money/power) or violence.
blueblimp 3y ago
There is a way, in my opinion: distribute AI widely and give it a diversity of values, so that any one AI attempting takeover (or being misused) is opposed by the others. This is best achieved by having both open source and a competitive market of many companies with their own proprietary models.
- ChatGTP 3y ago
  How do you give "AI" a diversity of values?
  - drvdevd 3y ago
    By driving down the costs of training and inference, and then encouraging experiments. For LLMs, QLoRA is arguably a great step in this direction.
  - blueblimp 3y ago
    Personalization, customization, etc.: by aligning AI systems to many users, we benefit from the already-existing diversity of values among different people. This could be achieved via open source or proprietary means; the important thing is that the system works for the user and not for whichever company made it.
- q1w2 3y ago
  Like giving nuclear weapons to everyone. What could go wrong?
  - blueblimp 3y ago
    AI isn't nuclear weaponry. It's bits, not atoms. It's more like encryption.
mtkhaos 3y ago
It's difficult as most of the risk can be reinterpreted as a highly advanced user. But that is where some form of hard personhood zero proof mechanism NEEDS to come in. This can then be used in conjunction with a Ledger used to track deployment of high spec models. And create an easy means to Audit and deploy new advanced tests to ensure safety. Really what everyone also need to keep in mind at the larger scale is that final turing test with no room for deniability. And remember all those Sci-fi movies and how that Moment is portrayed traditionally.
patrec 3y ago
> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of the strategic value nukes bring.
- nicce 3y ago
  If you look the the world politics, basically if you hold enough nuclear weapons, you can do whatever you want to those who don't have them. And based on the "dangers", new countries are prohibit to create them. And the countries which were quick enough to create them, holds all the power. Their value is immeasurable especially for the Russia. Without them, they could not attack to Ukraine. > non-proliferation treaties were a scam And yes, they mostly are right now. Russia has backed from them. There are no real consequences if you are backing off, and you can do it in any time. The parent commenter is most likely saying, that now the selected parties hold the power of AI, they want to prevent others to gain similar power, while maintaining all the value by themselves.
  - staunton 3y ago
    > There are no real consequences if you are backing off, and you can do it in any time. That's not quite true. Sure, noone is going to start a war about such a withdrawal. However, nuclear arsenals are expensive to maintain and it's even more expensive to be in an arms race. Also, nobody wants to risk nuclear war if they can avoid it. Civilian populations will support disarmament in times where they don't feel directly threatened. That's why lot of leaders of all persuasions have advocated for and taken part in efforts to reduce their arsenals. Same goes for relations between countries generally and the huge economic benefits that come with trade and cooperation. Withdrawing from nuclear treaties endangers all of these benefits and increases risk. A country would only choose this route out of desperation or for likely immediate gain.
    - 3y ago
sebzim4500 3y ago
General research into AI alignment does not require that those models are controlled by few corporations. On the contrary, the research would be easier with freely available very capable models. This is only helpful in that a superintelligence well aligned to make Sam Altman money is preferable to a superintelligence badly aligned that ends up killing humanity. It is fully possible that a well aligned (with its creators) superintelligence is still a net negative for humanity.
- mordae 3y ago
  If you consider a broader picture, unleashing a paperclip-style cripple AI (aligned to rising $MEGACORP profit) on the Local Group is almost definitely worse for all Local Group inhabitants than annihilating ourselves and not doing that.
toss1 3y ago
Yup. While I'm not on this "who's-who" panel of experts, I call bullshit. AI does present a range theoretical possibilities for existential doom, from teh "gray goo" and "paperclip optimizer" scenarios to Bostrom's post-singularity runaway self-improving superintelligence. I do see this as a genuine theoretical concern that could even potentially even be the Great Filter. However, the actual technology extant or even on the drawing boards today is nothing even on the same continent as those threats. We have a very vast ( and expensive) sets of probability-of-occurrence vectors that amount to a fancy parlor trick that produces surprising and sometimes useful results. While some tout the clustering of vectors around certain sets of words as implementing artificial creation of concepts, it's really nothing more than an advanced thesaurus; there is no evidence of concepts being weilded in relation to reality, tested for truth/falsehood value, etc. In fact, the machines are notorious and hilarious for hallucinating with a highly confident tone. We've created nothing more than a mirror of human works, and it displays itself as an industrial-scale bullshit artist (where bullshit is defined as expressions made to impress without care one way or the other for truth value). Meanwhile, this panel of experts makes this proclamation with not the slightest hint of what type of threat is present that would require any urgent attention, only that some threat exists that is on the scale of climate change. They mention no technological existential threat (e.g., runaway superintelligence), nor any societal threat (deepfakes, inherent bias, etc.). This is left as an exercise for the reader. What is the actual threat? It is most likely described in the Google "We Have No Moat" memo[0]. Basically, once AI is out there, these billionaires have no natural way to protect their income and create a scaleable way to extract money from the masses, UNLESS they get cooperation from politicians to prevent any competition from arising. As one of those billionaires, Peter Theil, said: "Competition is for losers" [1]. Since they have not yet figured out a way to cut out the competition using their advantages in leading the technology or their advantages in having trillions of dollars in deployable capital, they are seeking a legislated advantage. Bullshit. It must be ignored. [0] https://www.semianalysis.com/p/google-we-have-no-moat-and-neither https://www.semianalysis.com/p/google-we-have-no-moat-and-ne... [1] https://www.wsj.com/articles/peter-thiel-competition-is-for-losers-1410535536 https://www.wsj.com/articles/peter-thiel-competition-is-for-...
gfodor 3y ago
Here's my proposal: https://gfodor.medium.com/to-de-risk-ai-the-government-must-accelerate-knowledge-production-49c4f3c26aa0 https://gfodor.medium.com/to-de-risk-ai-the-government-must-... tl;dr: significant near term AI risk is real and comes from the capacity for imagined ideas, good and evil, to be autonomously executed on by agentic AI, not emergent superintelligent aliens. To de-risk this, we need to align AI quickly, which requires producing new knowledge. To accelerate the production of this knowledge, the government should abandon decelerationist policies and incentivize incremental alignment R&D by AI companies. And, critically, a new public/private research institution should be formed that grants privileged, fully funded investigators multi-year funding cycles with total scientific freedom and access to all state-of-the-art artificial intelligence systems operating under US law to maximize AI as a force multiplier in their research.
Animats 3y ago
> I have yet to see a solution for “AI safety” that doesn’t involve ceding control of our most powerful models to a small handful of corporations. That's an excellent point. Most of the near-term risks with AI involve corporations and governments acquiring more power. AI provides power tools for surveillance, oppression, and deception at scale. Those are already deployed and getting better. This mostly benefits powerful organizations. This alarm about strong AI taking over is a diversion from the real near-term threat. With AI, Big Brother can watch everything all the time. Listen to and evaluate everything you say and do. The cops and your boss already have some of that capability. Is something watching you right now through your webcam? Is something listening to you right now through your phone? Are you sure?
NumberWangMan 3y ago
Ok, so if we take AI safety / AI existential risk as real and important, there are two possibilities: 1) The only way to be safe is to cede control to the most powerful models to a small group (highly regulated corporations or governments) that can be careful. 2) There is a way to make AI safe without doing this. If 1 is true, then... sorry, I know it's not a very palatable solution, and may suck, but if that's all we've got I'll take it. If 2 is true, great. But it seems less likely than 1, to me. The important thing is not to unconsciously do some motivated reasoning, and think that AGI existential risk can't be a big deal, because if it is, that would mean that we have to cede control over to a small group of people to prevent disaster, which would suck, so there must be something else going on, like these people just want power.
- Darkphibre 3y ago
  I just don't see how the genie is put back in the bottle. Optimizations and new techniques are coming in at a breakneck pace, allowing for models that can run on consumer hardware.
  - NumberWangMan 3y ago
    I think it could be done. Or rather, instead of putting the genie back in the bottle, we could slow it down enough that we figure out how to ask it for wishes in a way that avoids all the monkey-paw's scenarios. Dropping the metaphor, running today's models isn't dangerous. We could criminalize developing stronger ones, and make a "Manhattan project" for AI aimed at figuring out how to not ruin the world with it. I think a big problem is what you point out -- once it's out, it's hard to prevent misuse. One bad AGI could end up making a virus that does massive damage to humanity. We might end up deciding that this tech is just too dangerous to be allowed to happen at all, at least until after humanity manages to digitize all our brains or something. But it's better to try to slow down as much as we can, for as long as we can, than to give up right from the get-go and wing it. Honestly, if it turns out that China ends up developing unsafe AI before we develop safe AI, I doubt it would have turned out much better for the average American if America were the ones to develop unsafe AI first. And if they cut corners and still manage to make safe AI and take over the world, that still sounds a heck of a lot better than anyone making unsafe AI.
hiAndrewQuinn 3y ago
I have one: Levy fines on actors judged to be attempting to extend AI capabilities beyond the current state of the art, and pay the fine to those private actors who prosecute them. https://www.overcomingbias.com/p/privately-enforced-punished-crimehtml https://www.overcomingbias.com/p/privately-enforced-punished...

by valine 3y ago

circuit10 3y ago

We don’t really have a good solution, I guess that’s why we need more research into it Companies might argue that giving them control might help but I don’t think most individuals working on it think that will work

sycamoretrees 3y ago
Is more research really going to offer any true solutions? I’d be genuinely interested in hearing about what research could potentially offer (the development of tools to counter AI disinformation? A deeper understanding of how LLMs work?), but it seems to me that the only “real” solution is ultimately political. The issue is that it would require elements of authoritarianism and censorship.
- wongarsu 3y ago
  A lot of research about avoiding extinction by AI is about alignment. LLMs are pretty harmless in that they (currently) don't have any goals, they just produce text. But at some point we will succeed in turning them into "thinking" agents that try to achieve a goal. Similar to a chess AI, but interacting with the real world instead. One of the big problems with that is that we don't have a good way to make sure the goals of the AI match what we want it to do. Even if the whole "human governance" political problem were solved, we still couldn't reliably control any AI. Solving that is a whole research field. Building better ways to understand the inner workings of neural networks is definitely one avenue
  - sycamoretrees 3y ago
    I see. Thanks for the reply. But I wonder if that’s not a bit too optimistic and not concrete enough. Alignment won’t solve the world’s woes, just like “enlightenment” (a word which sounds a lot like alignment and which is similarly undefinable) does not magically rectify the realities of the world. Why should bad actors care about alignment? Another example is climate change. We have a lot of good ideas which, combined, would stop us from killing millions of people across the world. We have the research - is more “research” really the key?
  - pixl97 3y ago
    Intelligence cannot be 'solved', I would go on to further say that an intelligence without the option of violence isn't an intelligence at all. If you suddenly wanted to kill people, for example, then could probably kill a few before you were stopped. That is typically the limits of an individuals power. Now, if you were a corporation with money, depending on the strategy you used you could likely kill anywhere from hundreds to hundreds of thousands. Kick it up to government level, and well, the term "just a statistic" exists for a reason. We tend to have laws around these behaviors, but they are typically punitive. The law realizes that humans, and human systems will unalign themselves from "moral" behavior (whatever that may be considered at the time). When the lawgiver itself becomes unaligned, well, things tend to get bad. Human alignment typically consists of benefits (I give you nice things/money/power) or violence.

blueblimp 3y ago

There is a way, in my opinion: distribute AI widely and give it a diversity of values, so that any one AI attempting takeover (or being misused) is opposed by the others. This is best achieved by having both open source and a competitive market of many companies with their own proprietary models.

ChatGTP 3y ago
How do you give "AI" a diversity of values?
- drvdevd 3y ago
  By driving down the costs of training and inference, and then encouraging experiments. For LLMs, QLoRA is arguably a great step in this direction.
- blueblimp 3y ago
  Personalization, customization, etc.: by aligning AI systems to many users, we benefit from the already-existing diversity of values among different people. This could be achieved via open source or proprietary means; the important thing is that the system works for the user and not for whichever company made it.
q1w2 3y ago
Like giving nuclear weapons to everyone. What could go wrong?
- blueblimp 3y ago
  AI isn't nuclear weaponry. It's bits, not atoms. It's more like encryption.

mtkhaos 3y ago

It's difficult as most of the risk can be reinterpreted as a highly advanced user. But that is where some form of hard personhood zero proof mechanism NEEDS to come in. This can then be used in conjunction with a Ledger used to track deployment of high spec models. And create an easy means to Audit and deploy new advanced tests to ensure safety. Really what everyone also need to keep in mind at the larger scale is that final turing test with no room for deniability. And remember all those Sci-fi movies and how that Moment is portrayed traditionally.

patrec 3y ago

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of the strategic value nukes bring.

nicce 3y ago
If you look the the world politics, basically if you hold enough nuclear weapons, you can do whatever you want to those who don't have them. And based on the "dangers", new countries are prohibit to create them. And the countries which were quick enough to create them, holds all the power. Their value is immeasurable especially for the Russia. Without them, they could not attack to Ukraine. > non-proliferation treaties were a scam And yes, they mostly are right now. Russia has backed from them. There are no real consequences if you are backing off, and you can do it in any time. The parent commenter is most likely saying, that now the selected parties hold the power of AI, they want to prevent others to gain similar power, while maintaining all the value by themselves.
- staunton 3y ago
  > There are no real consequences if you are backing off, and you can do it in any time. That's not quite true. Sure, noone is going to start a war about such a withdrawal. However, nuclear arsenals are expensive to maintain and it's even more expensive to be in an arms race. Also, nobody wants to risk nuclear war if they can avoid it. Civilian populations will support disarmament in times where they don't feel directly threatened. That's why lot of leaders of all persuasions have advocated for and taken part in efforts to reduce their arsenals. Same goes for relations between countries generally and the huge economic benefits that come with trade and cooperation. Withdrawing from nuclear treaties endangers all of these benefits and increases risk. A country would only choose this route out of desperation or for likely immediate gain.
  - 3y ago

sebzim4500 3y ago

General research into AI alignment does not require that those models are controlled by few corporations. On the contrary, the research would be easier with freely available very capable models. This is only helpful in that a superintelligence well aligned to make Sam Altman money is preferable to a superintelligence badly aligned that ends up killing humanity. It is fully possible that a well aligned (with its creators) superintelligence is still a net negative for humanity.

mordae 3y ago
If you consider a broader picture, unleashing a paperclip-style cripple AI (aligned to rising $MEGACORP profit) on the Local Group is almost definitely worse for all Local Group inhabitants than annihilating ourselves and not doing that.

toss1 3y ago

Yup. While I'm not on this "who's-who" panel of experts, I call bullshit. AI does present a range theoretical possibilities for existential doom, from teh "gray goo" and "paperclip optimizer" scenarios to Bostrom's post-singularity runaway self-improving superintelligence. I do see this as a genuine theoretical concern that could even potentially even be the Great Filter. However, the actual technology extant or even on the drawing boards today is nothing even on the same continent as those threats. We have a very vast ( and expensive) sets of probability-of-occurrence vectors that amount to a fancy parlor trick that produces surprising and sometimes useful results. While some tout the clustering of vectors around certain sets of words as implementing artificial creation of concepts, it's really nothing more than an advanced thesaurus; there is no evidence of concepts being weilded in relation to reality, tested for truth/falsehood value, etc. In fact, the machines are notorious and hilarious for hallucinating with a highly confident tone. We've created nothing more than a mirror of human works, and it displays itself as an industrial-scale bullshit artist (where bullshit is defined as expressions made to impress without care one way or the other for truth value). Meanwhile, this panel of experts makes this proclamation with not the slightest hint of what type of threat is present that would require any urgent attention, only that some threat exists that is on the scale of climate change. They mention no technological existential threat (e.g., runaway superintelligence), nor any societal threat (deepfakes, inherent bias, etc.). This is left as an exercise for the reader. What is the actual threat? It is most likely described in the Google "We Have No Moat" memo[0]. Basically, once AI is out there, these billionaires have no natural way to protect their income and create a scaleable way to extract money from the masses, UNLESS they get cooperation from politicians to prevent any competition from arising. As one of those billionaires, Peter Theil, said: "Competition is for losers" [1]. Since they have not yet figured out a way to cut out the competition using their advantages in leading the technology or their advantages in having trillions of dollars in deployable capital, they are seeking a legislated advantage. Bullshit. It must be ignored. [0] https://www.semianalysis.com/p/google-we-have-no-moat-and-neither https://www.semianalysis.com/p/google-we-have-no-moat-and-ne... [1] https://www.wsj.com/articles/peter-thiel-competition-is-for-losers-1410535536 https://www.wsj.com/articles/peter-thiel-competition-is-for-...

gfodor 3y ago

Here's my proposal: https://gfodor.medium.com/to-de-risk-ai-the-government-must-accelerate-knowledge-production-49c4f3c26aa0 https://gfodor.medium.com/to-de-risk-ai-the-government-must-... tl;dr: significant near term AI risk is real and comes from the capacity for imagined ideas, good and evil, to be autonomously executed on by agentic AI, not emergent superintelligent aliens. To de-risk this, we need to align AI quickly, which requires producing new knowledge. To accelerate the production of this knowledge, the government should abandon decelerationist policies and incentivize incremental alignment R&D by AI companies. And, critically, a new public/private research institution should be formed that grants privileged, fully funded investigators multi-year funding cycles with total scientific freedom and access to all state-of-the-art artificial intelligence systems operating under US law to maximize AI as a force multiplier in their research.

Animats 3y ago

> I have yet to see a solution for “AI safety” that doesn’t involve ceding control of our most powerful models to a small handful of corporations. That's an excellent point. Most of the near-term risks with AI involve corporations and governments acquiring more power. AI provides power tools for surveillance, oppression, and deception at scale. Those are already deployed and getting better. This mostly benefits powerful organizations. This alarm about strong AI taking over is a diversion from the real near-term threat. With AI, Big Brother can watch everything all the time. Listen to and evaluate everything you say and do. The cops and your boss already have some of that capability. Is something watching you right now through your webcam? Is something listening to you right now through your phone? Are you sure?

NumberWangMan 3y ago

Ok, so if we take AI safety / AI existential risk as real and important, there are two possibilities: 1) The only way to be safe is to cede control to the most powerful models to a small group (highly regulated corporations or governments) that can be careful. 2) There is a way to make AI safe without doing this. If 1 is true, then... sorry, I know it's not a very palatable solution, and may suck, but if that's all we've got I'll take it. If 2 is true, great. But it seems less likely than 1, to me. The important thing is not to unconsciously do some motivated reasoning, and think that AGI existential risk can't be a big deal, because if it is, that would mean that we have to cede control over to a small group of people to prevent disaster, which would suck, so there must be something else going on, like these people just want power.

Darkphibre 3y ago
I just don't see how the genie is put back in the bottle. Optimizations and new techniques are coming in at a breakneck pace, allowing for models that can run on consumer hardware.
- NumberWangMan 3y ago
  I think it could be done. Or rather, instead of putting the genie back in the bottle, we could slow it down enough that we figure out how to ask it for wishes in a way that avoids all the monkey-paw's scenarios. Dropping the metaphor, running today's models isn't dangerous. We could criminalize developing stronger ones, and make a "Manhattan project" for AI aimed at figuring out how to not ruin the world with it. I think a big problem is what you point out -- once it's out, it's hard to prevent misuse. One bad AGI could end up making a virus that does massive damage to humanity. We might end up deciding that this tech is just too dangerous to be allowed to happen at all, at least until after humanity manages to digitize all our brains or something. But it's better to try to slow down as much as we can, for as long as we can, than to give up right from the get-go and wing it. Honestly, if it turns out that China ends up developing unsafe AI before we develop safe AI, I doubt it would have turned out much better for the average American if America were the ones to develop unsafe AI first. And if they cut corners and still manage to make safe AI and take over the world, that still sounds a heck of a lot better than anyone making unsafe AI.

hiAndrewQuinn 3y ago

I have one: Levy fines on actors judged to be attempting to extend AI capabilities beyond the current state of the art, and pay the fine to those private actors who prosecute them. https://www.overcomingbias.com/p/privately-enforced-punished-crimehtml https://www.overcomingbias.com/p/privately-enforced-punished...