4 ms·

It says "will seek to open source technology for the public benefit when applicable" they have open sourced a number of things, Whisper most notably. Nothing ab

It says "will seek to open source technology for the public benefit when applicable" they have open sourced a number of things, Whisper most notably. Nothing about that is a promise to open source everything and they just need to say it wasn't applicable for ChatGPT or DallE because of safety.

stubish 3y ago
I doubt the safety argument will hold up in court. Anything safe enough to allow Microsoft or others access too would be safe enough to release publicly. Our AI overlords are not going to respect an NDA. And for the public safety/disinformation side of things, I think it is safe to say that cat is out of the bag and chasing the horse that has bolted.
- sanxiyn 3y ago
  I am unsure. You can't (for example) fine tune over API. Is anything safe for Microsoft to fine tune really safe for Russia, CCP, etc. to fine tune? Open weight (which I think is more accurate term than open source here) models enable both much more actors and much more actions than the current status.
  - pclmulqdq 3y ago
    You can fine tune over the API. Also, Russia and the CCP likely have the model weights. They probably have spies in OpenAI or Microsoft with access to the weights.
    - WanderPanda 3y ago
      Interesting thought experiment! How would they best take advantage of the weights and what would be signs/actions that we could observe that signal it is likely they have the weights?
      - simfree 3y ago
        We know Microsoft experienced a full breach of Office 365/Microsoft 365 and Azure infrastructure by a nation state actor: https://www.imprivata.com/blog/strengthening-security-5-lessons-learn-microsofts-recent-azure-and-exchange-server-challenges https://www.imprivata.com/blog/strengthening-security-5-less...
    - HeavyStorm 3y ago
      I don't think such speculation would _hold in court_
      - pclmulqdq 3y ago
        A quick Google search has confirmed that Microsoft has confirmed at least the Russia part: https://www.cyberark.com/resources/blog/apt29s-attack-on-microsoft-tracking-cozy-bears-footprints https://www.cyberark.com/resources/blog/apt29s-attack-on-mic... It's very hard to argue that when you give 100,000 people access to materials that are inherently worth billions, none of them are stealing those materials. Google has enough leakers to conservative media of all places that you should suspect that at least one Googler is exfiltrating data to China, Russia, or India.
    - petre 3y ago
      They'll train it on Xi Jingping Thought so that the people of China can move on with their lives and use the Xi bot instead of wasting precious man hours actually studying the texts. The Russians will obviously use it to spread Kremlin's narratives on the Internet in all languages, including Klingon and Elvish.
- bandyaboot 3y ago
  If the above statement is the only “commitment” they’ve made to open-source, then that argument won’t need to be made in court. They just need to reference the vague language that basically leaves the door open to do anything they want.
- tintor 3y ago
  `Anything safe enough to allow Microsoft or others access too would be safe enough to release publicly.` This makes absolutely no sense.
  - rl3 3y ago
    >This makes absolutely no sense. https://news.ycombinator.com/item?id=34716375 https://news.ycombinator.com/item?id=34716375 What about now?
    - cutemonster 3y ago
      Microsoft doesn't run troll farms trying to manipulate the voters to change the US to a dictatorship, or develop killer drone swarms or have nukes. (Not saying OpenAI isn't greedy)
      - salawat 3y ago
        ...What OS do you think many of these places use? Linux is still niche af. In a real, tangible way, it may very well be the case that yes, Microsoft does, in fact, run them.
        Dylan16807 3y ago
        The "..." is not warranted because that is clearly not the sense of "run" they were talking about.
      - rl3 3y ago
        I think you make a good point. My argument was that Microsoft's security isn't that great, therefore the risk of the model ending up in the hands of the bad actors you mention isn't sufficiently low.
        cutemonster 3y ago
        Aha, ok thanks for explaining
- WhatIsAModel 3y ago
  https://arxiv.org/abs/2311.03348 https://arxiv.org/abs/2311.03348 This seems to make a decent argument that these models are potentially not safe. I prefer criminals don't have access to a PhD bomb making assistants who can explain the process to them like they are 12. While the cat may be out of the bag, you don't just hand out guns to everyone (for free) because a few people misused them.
thayne 3y ago
I think that position would be a lot more defensible if they weren't giving another for-profit company access to it. And there is definitely a conflict of interest when not revealing the source gives them a competitive advantage in selling their product. There's also the question of if the source is too dangerous to make public, how can they be sure the final product is safe? An argument could be made it isn't safe.
- theptip 3y ago
  It’s easy to defend this position. It is safer to operate an AI in a centralized service, because if you discover dangerous capabilities you can turn it off or mitigate them. If you open-weight the model, if dangerous capabilities are later discovered there is no way to put the genie back in the bottle; the weights are out there, anyone can use them. This of course applies to both mundane harms (eg generating deepfake porn of famous people) or existential risks (eg power-seeking behavior).
  - WalterSear 3y ago
    This was all obvious >before< they wrote the charter.
    - theptip 3y ago
      I don’t think this belief was widespread at all at that time. Indeed, it’s not widespread even now, lots of folks round here are still confused by “open weight sounds like open source and we like open source”, and Elon is still charging towards fully open models. (In general I think if you are more worried about a baby machine god owned and aligned by Meta than complete annihilation from unaligned ASI then you’ll prefer open weights no matter the theoretical risk.)
      - WalterSear 3y ago
        IMHE, it's been part of widespread discussions in the AI research and AI safety communities since the 2000s.
canjobear 3y ago
The "when applicable" gets them out of nearly anything.
man4 3y ago
[dead]

stubish 3y ago

I doubt the safety argument will hold up in court. Anything safe enough to allow Microsoft or others access too would be safe enough to release publicly. Our AI overlords are not going to respect an NDA. And for the public safety/disinformation side of things, I think it is safe to say that cat is out of the bag and chasing the horse that has bolted.

sanxiyn 3y ago
I am unsure. You can't (for example) fine tune over API. Is anything safe for Microsoft to fine tune really safe for Russia, CCP, etc. to fine tune? Open weight (which I think is more accurate term than open source here) models enable both much more actors and much more actions than the current status.
- pclmulqdq 3y ago
  You can fine tune over the API. Also, Russia and the CCP likely have the model weights. They probably have spies in OpenAI or Microsoft with access to the weights.
  - WanderPanda 3y ago
    Interesting thought experiment! How would they best take advantage of the weights and what would be signs/actions that we could observe that signal it is likely they have the weights?
    - simfree 3y ago
      We know Microsoft experienced a full breach of Office 365/Microsoft 365 and Azure infrastructure by a nation state actor: https://www.imprivata.com/blog/strengthening-security-5-lessons-learn-microsofts-recent-azure-and-exchange-server-challenges https://www.imprivata.com/blog/strengthening-security-5-less...
  - HeavyStorm 3y ago
    I don't think such speculation would _hold in court_
    - pclmulqdq 3y ago
      A quick Google search has confirmed that Microsoft has confirmed at least the Russia part: https://www.cyberark.com/resources/blog/apt29s-attack-on-microsoft-tracking-cozy-bears-footprints https://www.cyberark.com/resources/blog/apt29s-attack-on-mic... It's very hard to argue that when you give 100,000 people access to materials that are inherently worth billions, none of them are stealing those materials. Google has enough leakers to conservative media of all places that you should suspect that at least one Googler is exfiltrating data to China, Russia, or India.
  - petre 3y ago
    They'll train it on Xi Jingping Thought so that the people of China can move on with their lives and use the Xi bot instead of wasting precious man hours actually studying the texts. The Russians will obviously use it to spread Kremlin's narratives on the Internet in all languages, including Klingon and Elvish.
bandyaboot 3y ago
If the above statement is the only “commitment” they’ve made to open-source, then that argument won’t need to be made in court. They just need to reference the vague language that basically leaves the door open to do anything they want.
tintor 3y ago
`Anything safe enough to allow Microsoft or others access too would be safe enough to release publicly.` This makes absolutely no sense.
- rl3 3y ago
  >This makes absolutely no sense. https://news.ycombinator.com/item?id=34716375 https://news.ycombinator.com/item?id=34716375 What about now?
  - cutemonster 3y ago
    Microsoft doesn't run troll farms trying to manipulate the voters to change the US to a dictatorship, or develop killer drone swarms or have nukes. (Not saying OpenAI isn't greedy)
    - salawat 3y ago
      ...What OS do you think many of these places use? Linux is still niche af. In a real, tangible way, it may very well be the case that yes, Microsoft does, in fact, run them.
      - Dylan16807 3y ago
        The "..." is not warranted because that is clearly not the sense of "run" they were talking about.
    - rl3 3y ago
      I think you make a good point. My argument was that Microsoft's security isn't that great, therefore the risk of the model ending up in the hands of the bad actors you mention isn't sufficiently low.
      - cutemonster 3y ago
        Aha, ok thanks for explaining
WhatIsAModel 3y ago
https://arxiv.org/abs/2311.03348 https://arxiv.org/abs/2311.03348 This seems to make a decent argument that these models are potentially not safe. I prefer criminals don't have access to a PhD bomb making assistants who can explain the process to them like they are 12. While the cat may be out of the bag, you don't just hand out guns to everyone (for free) because a few people misused them.

thayne 3y ago

I think that position would be a lot more defensible if they weren't giving another for-profit company access to it. And there is definitely a conflict of interest when not revealing the source gives them a competitive advantage in selling their product. There's also the question of if the source is too dangerous to make public, how can they be sure the final product is safe? An argument could be made it isn't safe.

theptip 3y ago
It’s easy to defend this position. It is safer to operate an AI in a centralized service, because if you discover dangerous capabilities you can turn it off or mitigate them. If you open-weight the model, if dangerous capabilities are later discovered there is no way to put the genie back in the bottle; the weights are out there, anyone can use them. This of course applies to both mundane harms (eg generating deepfake porn of famous people) or existential risks (eg power-seeking behavior).
- WalterSear 3y ago
  This was all obvious >before< they wrote the charter.
  - theptip 3y ago
    I don’t think this belief was widespread at all at that time. Indeed, it’s not widespread even now, lots of folks round here are still confused by “open weight sounds like open source and we like open source”, and Elon is still charging towards fully open models. (In general I think if you are more worried about a baby machine god owned and aligned by Meta than complete annihilation from unaligned ASI then you’ll prefer open weights no matter the theoretical risk.)
    - WalterSear 3y ago
      IMHE, it's been part of widespread discussions in the AI research and AI safety communities since the 2000s.

canjobear 3y ago

The "when applicable" gets them out of nearly anything.

man4 3y ago

[dead]