4 ms·

Ben Thompson has the best take on this (if a bit biased against nonprofits): https://stratechery.com/2023/openais-misalignment-and-microsofts-gain/ https://str

by fmajid 3y ago

Ben Thompson has the best take on this (if a bit biased against nonprofits): https://stratechery.com/2023/openais-misalignment-and-microsofts-gain/ https://stratechery.com/2023/openais-misalignment-and-micros... I don't know what the risk of AI is, but having a nonprofit investigate solutions to prevent them is a worthwhile pursuit, as for-profit corporations will not do it (as shown by the firing of Timnit Gebru and Margaret Mitchell by Google). If they really believe in that mission, they should develop guardrails technology and open-source it so the companies like Microsoft, Google, Meta, Amazon et al who are certainly not investing in AI safety but won't mind using others' work for free can inegrate it. But that's not going to be lucrative and that's why most OpenAI employees will leave for greener pastures.

RcouF1uZ4gsC 3y ago
> but having a nonprofit investigate solutions to prevent them is a worthwhile pursuit, This is forgetting that power is an even greater temptation than money. The non-profits will all come up with solutions that have them serving as gatekeepers, to keep the unwashed masses from accessing something that that is too dangerous for the common person. I would rather have for-profit corporations control it, rather that non-profits. Ideally, Inwould like it to be open sourced so that the common person could control and align AI with their own goals.
- kibwen 3y ago
  > I would rather have for-profit corporations control it, rather that non-profits. The problem isn't the profit model, the problem is the ability to unilaterally exercise power, which is just as much of a risk with the way that most for-profit companies are structured as top-down dictatorships. There's no reason to trust for-profit companies to do anything other than attempt to maximize profit, even if that destroys everything around them in the process.
- fmajid 3y ago
  There is no profit in AI safety, just as cars did not have seat belts until Ralph Nader effectively forced them to by publishing Unsafe at any Speed. For-profit corporations have zero interest in controlling something that is not profitable, unless in conjunction with captured regulation it helps them keep challengers out. If it's open-sourced, it doesn't matter who wrote it as long as they are economically sustainable.
  - dmix 3y ago
    > There is no profit in AI safety AI safety is barely even a tangible thing to measure like that. It's mostly just fears and a lose set of ideas for a hypothetical future AGI that we're not even close to. So far OpenAI's "controls" it's just increasingly expanding the list of no-no things topics and some philosophy work around iRobot type rules. They also slow walked the release of GPT because of fears of misinformation, spam, and deepfakey stuff that never really materialized. Most proposals for safety is just "slowing development" of mostly LLMs, calls for vague gov regulation, or hand wringing over commercialization. The commercialization thing is most controversial because OpenAI claimed to be open and non-profit. But even with that the correlation between less-commercialization == more safety is not clear, other than prioritizing what OpenAI's team spends their time doing. Which again is hard to tangibly measure what that realistically means for 'safety' in the near term.
  - pawelmurias 3y ago
    > There is no profit in AI safety An AI that does what it is told too seems both way more profitable and safer.
    - justaj 3y ago
      I'm guessing the issues will lie in cases where it appears to be doing what it's told, but it would only pretend to be doing so (with no obvious way to tell)
Always_Anon 3y ago
>(as shown by the firing of Timnit Gebru...) Timnit Gebru was fired for being a toxic /r/ImTheMainCharacter SJW that was enshittifiy the entire AI/ML department. Management correctly fired someone that was holding an entire department hostage in her crusade against the grievance de jure.
- victor106 3y ago
  Agree 100% with this
- astrange 3y ago
  She was fired for threatening to quit. If you threaten something like that it just happens; you can't stop the machinery.
  - Always_Anon 3y ago
    I wish more of them did that. She gave them the excuse they were looking for. She thought she was being smart, instead she self-owned herself in the most spectacular fashion.
- VirusNewbie 3y ago
  I'm at Google, I 100% agree with this. Also her paper was garbage. You can maybe get away with being a self righteous prick or an outright asshole if you are brilliant, but it's clear by reading her work she didn't fall into that category.
danbmil 3y ago
Agreed. This discussion around safety reminds me of the early days of cybersecurity, when security by obscurity was the norm. It's counter-intuitive, but locking up a technology is like trying to control prices and wages. It just doesn't work -- unless you confiscate every GPU in the world and bomb datacenters etc. The best way to align with the coming AGI's and ASI's is to build them in the sunlight. Every lock-em-up approach is doomed to fail (I guess that makes me a meta-doomer?)

by fmajid 3y ago

RcouF1uZ4gsC 3y ago

> but having a nonprofit investigate solutions to prevent them is a worthwhile pursuit, This is forgetting that power is an even greater temptation than money. The non-profits will all come up with solutions that have them serving as gatekeepers, to keep the unwashed masses from accessing something that that is too dangerous for the common person. I would rather have for-profit corporations control it, rather that non-profits. Ideally, Inwould like it to be open sourced so that the common person could control and align AI with their own goals.

kibwen 3y ago
> I would rather have for-profit corporations control it, rather that non-profits. The problem isn't the profit model, the problem is the ability to unilaterally exercise power, which is just as much of a risk with the way that most for-profit companies are structured as top-down dictatorships. There's no reason to trust for-profit companies to do anything other than attempt to maximize profit, even if that destroys everything around them in the process.
fmajid 3y ago
There is no profit in AI safety, just as cars did not have seat belts until Ralph Nader effectively forced them to by publishing Unsafe at any Speed. For-profit corporations have zero interest in controlling something that is not profitable, unless in conjunction with captured regulation it helps them keep challengers out. If it's open-sourced, it doesn't matter who wrote it as long as they are economically sustainable.
- dmix 3y ago
  > There is no profit in AI safety AI safety is barely even a tangible thing to measure like that. It's mostly just fears and a lose set of ideas for a hypothetical future AGI that we're not even close to. So far OpenAI's "controls" it's just increasingly expanding the list of no-no things topics and some philosophy work around iRobot type rules. They also slow walked the release of GPT because of fears of misinformation, spam, and deepfakey stuff that never really materialized. Most proposals for safety is just "slowing development" of mostly LLMs, calls for vague gov regulation, or hand wringing over commercialization. The commercialization thing is most controversial because OpenAI claimed to be open and non-profit. But even with that the correlation between less-commercialization == more safety is not clear, other than prioritizing what OpenAI's team spends their time doing. Which again is hard to tangibly measure what that realistically means for 'safety' in the near term.
- pawelmurias 3y ago
  > There is no profit in AI safety An AI that does what it is told too seems both way more profitable and safer.
  - justaj 3y ago
    I'm guessing the issues will lie in cases where it appears to be doing what it's told, but it would only pretend to be doing so (with no obvious way to tell)

Always_Anon 3y ago

>(as shown by the firing of Timnit Gebru...) Timnit Gebru was fired for being a toxic /r/ImTheMainCharacter SJW that was enshittifiy the entire AI/ML department. Management correctly fired someone that was holding an entire department hostage in her crusade against the grievance de jure.

victor106 3y ago
Agree 100% with this
astrange 3y ago
She was fired for threatening to quit. If you threaten something like that it just happens; you can't stop the machinery.
- Always_Anon 3y ago
  I wish more of them did that. She gave them the excuse they were looking for. She thought she was being smart, instead she self-owned herself in the most spectacular fashion.
VirusNewbie 3y ago
I'm at Google, I 100% agree with this. Also her paper was garbage. You can maybe get away with being a self righteous prick or an outright asshole if you are brilliant, but it's clear by reading her work she didn't fall into that category.

danbmil 3y ago

Agreed. This discussion around safety reminds me of the early days of cybersecurity, when security by obscurity was the norm. It's counter-intuitive, but locking up a technology is like trying to control prices and wages. It just doesn't work -- unless you confiscate every GPU in the world and bomb datacenters etc. The best way to align with the coming AGI's and ASI's is to build them in the sunlight. Every lock-em-up approach is doomed to fail (I guess that makes me a meta-doomer?)