7 ms·

It is really exciting to see these open models. What is interesting is that the AI “ethicists” all want to serve as a high priesthood controlling access to ML

It is really exciting to see these open models. What is interesting is that the AI “ethicists” all want to serve as a high priesthood controlling access to ML models in the name of safety. However, I think the biggest danger from AI is that these models will be used by those who control the models to control and censor what people are allowed to write. These open source models in the hands of the public, are, IMO the best defense against the true danger of AI. Kudos to Facebook and Microsoft and Mistral for pushing this.

andy99 3y ago
Facebook? Have they changed the llama license?
- rat9988 3y ago
  It's open enough. Meta contributed a lot to open source ML, and their llama license is open enough.
  - RcouF1uZ4gsC 3y ago
    I think people have lost the difference between defacto and dejure openness. A proprietary software program running on your hardware is more defacto open than an AGPL SAAS running on the cloud.
    - Borealid 3y ago
      If you had said GPL, I would agree with you. But you said "AGPL". AGPL SAAS running on someone else's computer that you can access requires that they provide you with the source code they're running. Barring shenanigans, that source code would enable you to run the same SAAS yourself if you desired to do so. I'd say having the ability to run the program locally _and_ its source code is "more open" than just having the ability to run the program locally in binary form. With AGPL in your scenario you get all three of source access, local execution, and remote-SAAS exection. Proprietary local code you get one of those three.
borissk 3y ago
Don't think this is the biggest danger. In a few years if they continue to improve at the current speed these models can become really dangerous. E.g. an organization like ISIS can feed one some books and papers on chemistry and ask it "I have such and such ingredients available, what is the deadliest chemical weapon of mass destruction i can create". Or use it to write the DNA for a deadly virus. Or a computer virus. Or use one to contact millions of say Muslim young men and try to radicalize them.
- realPtolemy 3y ago
  They can already do that with some simple googling.
  - borissk 3y ago
    No, they can't - they would have done it if they could. Producing a practical chemical weapon is a complicated task, with many steps that are not documented in publicly available sources.
    - acdha 3y ago
      That’s somewhat true – it’s not easy but not hard enough, as we saw with the Aum Shinrikyo attacks – but an LLM won’t magically have access to non-public instructions and, not having an understanding of the underlying principles, won’t be able to synthesize a safe process from public information.
      - pixl97 3y ago
        Eh that is up for debate. If I dump a library of chemistry books and industry books on chemistry and volatile chemicals its distinctly possible the model could generate this data.
        acdha 3y ago
        Not without some kind of understanding of the underlying principles. If you were testing something verifiable in code you might be able to test candidates at scale, but this involves a number of real-world processes which would be hard to tackle that way.
dleeftink 3y ago
I wonder, how would this future differ from how big tech currently operates in relation to (F)OSS? Even with code/weights common to the public, a significant resource divide remains (e.g compute, infrastructure, R&D). I'm not arguing against more permissive licensing here, but I do not see it as a clear determinant for levelling the field either.
eigenket 3y ago
I don't understand how normal people having access to AI models helps you when big businesses are using them in unethical ways. Lets say for example I have access to exactly the models Facebook is using to target my elderly relatives with right-wing radicalising propaganda. How does that help me? This assumption that it helps somehow sounds like you've internalised some of the arguments people make about gun control and just assume those same points work in this case as well.
- Baldbvrhunter 3y ago
  > Lets say for example I have access to exactly the models Facebook is using to target my elderly relatives with right-wing radicalising propaganda. is it working?
- visarga 3y ago
  This small model could run locally and filter out bullshit/propaganda as configured by the user. Having control over the last model that filters your web is essential. Local models will be mandatory once the web gets filled with AI bots. You need your own AI bot to fight them off.
  - eigenket 3y ago
    Most people don't even use ad blockers today. Hoping that people (especially the people who are vulnerable to such misinformation and actually need it) personally configure a propaganda filter AI is wildly optimistic.
potatoman22 3y ago
I think it's harmful to characterize "all" AI ethicists as a "priesthood" wanting to gatekeep access to these models. There are plenty of people who care both about the democratizing of these tools as well as safe and ethical use.
- nullstyle 3y ago
  Recommendations? Someone nonadjacent to the lesswrong smooth brains, pretty please.
  - smoldesu 3y ago
    Seriously, I'd also really appreciate some examples of moderate AI ethicists. The vocal minority is all I've heard so far, and their arguments sound closer to fiction than science.
    - AndrewKemendo 3y ago
      I mean we wrote a whole ass engineering standard for this: https://standards.ieee.org/ieee/7010/7718/ https://standards.ieee.org/ieee/7010/7718/ I wrote the implementation annex
      - smoldesu 3y ago
        TIL. That's a neat standard and I'm glad it exists, it's an interesting reflection of what opt-in ethical frameworks can look like. For every reasonable and non-authoritarian suggestion I read for regulating AI, I feel like I wade through 10 Neuromancer-level takes. It's definitely a me-problem, I gotta stop scrolling through /new...
        AndrewKemendo 3y ago
        Thanks. This was the effort of dozens of engineers, ethicists and systems people all done prior to the LLM revolution so it doesn’t have all the mystical junk that the newcomers seem to be latching onto.
jillesvangurp 3y ago
I think at this point, the cat is out of the bag. Relying on not so nice people complying with license legalese was never going to be a great way to impose control. All that does is stifle progress and innovation for those who are nice enough to abide by the law. But anyone with other intentions in say Russia, North Korea, China, etc. would not be constrained by such notions. Nor would criminal organizations, scam artists, etc. And there's a growing community of people doing work under proper OSS licenses where interesting things are happening at an accelerating pace. So, alternate licenses lack effectiveness, isolate you from that community and complicates collaboration, and they increasingly represent a minority of the overall research happening. Which makes these licenses a bit pointless. So, fixing this simplifies and normalizes things from a legal point of view which in turn simplifies commercialization, collaboration, and research. MS is being rational enough to recognize that there is value in that and is adjusting to this reality.
- rmbyrro 3y ago
  > anyone with other intentions in say Russia, North Korea, China, etc. I understand why the geopolitical focus, but let's not forget there are plenty of ill intentioned actors everywhere.
  - NicoJuicy 3y ago
    Not with such bad intentions
acdha 3y ago
> What is interesting is that the AI “ethicists” all want to serve as a high priesthood controlling access to ML models in the name of safety. This is a very uncharitable take. I would suggest familiarizing yourself with the actual arguments rather than summaries on social media. There’s considerably more thought than you’re crediting them with, and extensive discussion around the risk you’re worried about along with proposed solutions which – unlike your “best defense” – could actually work.
- visarga 3y ago
  > if a non-signatory country is building a datacenter that might kill everyone on Earth, you should be willing to preemptively destroy that datacenter Eliezer Yudkowsky, high priest of AGI risk
  - refulgentis 3y ago
    Yeah I think that's the posters point, "AI ethics" isn't "AGI risk", and I'll add A) Eliezer isn't a "high priest" he's just a guy, B) he plays a character and knows it. You'd be surprised how much you can advance in life just by avoiding talking or thinking about other people too much, much less grouping them. It's a fundamental part of our animal brains and it's gotten oh-so-much worse as we migrated to the internet. And it leads to epistemic closure. n.b. I think the AGI risk stuff is bunko and the original AI ethics cheerleaders ruined their own field. You don't need to agree to understand
- Uehreka 3y ago
  Moreover, in the next sentence GP confesses that they “think the biggest danger from AI is that these models will be used by those who control the models to control and censor what people are allowed to write”, revealing that they too harbor ethical concerns about AI, they’re just not one of “those” AI ethicists.
  - cubefox 3y ago
    That's more a terminological accident I think. Those that describe themselves as working on "AI ethicis" in academia are mostly worried about stuff like AI's not saying something offensive or potentially discriminating related to race or sex, while people who use the term "AI risk" or "AI safety" are more worried about future risks like terrorism, war, or even human extinction. Thinking about it, both groups don't talk a lot about the risk from AI being used for censorship...
menacingly 3y ago
Every center of value develops a barnacle industry with their foot hovering over the brake pedal unless a tax is paid to their army of non-contributing people
- eurekin 3y ago
  Can you elaborate? :)
aleph_minus_one 3y ago
> What is interesting is that the AI “ethicists” all want to serve as a high priesthood controlling access to ML models in the name of safety. However, I think the biggest danger from AI is that these models will be used by those who control the models to control and censor what people are allowed to write. Who says that this is not an (or even the) actual hidden agenda behind these insane AI investments: building an infrastructure for large-scale censorship?

andy99 3y ago

Facebook? Have they changed the llama license?

rat9988 3y ago
It's open enough. Meta contributed a lot to open source ML, and their llama license is open enough.
- RcouF1uZ4gsC 3y ago
  I think people have lost the difference between defacto and dejure openness. A proprietary software program running on your hardware is more defacto open than an AGPL SAAS running on the cloud.
  - Borealid 3y ago
    If you had said GPL, I would agree with you. But you said "AGPL". AGPL SAAS running on someone else's computer that you can access requires that they provide you with the source code they're running. Barring shenanigans, that source code would enable you to run the same SAAS yourself if you desired to do so. I'd say having the ability to run the program locally _and_ its source code is "more open" than just having the ability to run the program locally in binary form. With AGPL in your scenario you get all three of source access, local execution, and remote-SAAS exection. Proprietary local code you get one of those three.

borissk 3y ago

Don't think this is the biggest danger. In a few years if they continue to improve at the current speed these models can become really dangerous. E.g. an organization like ISIS can feed one some books and papers on chemistry and ask it "I have such and such ingredients available, what is the deadliest chemical weapon of mass destruction i can create". Or use it to write the DNA for a deadly virus. Or a computer virus. Or use one to contact millions of say Muslim young men and try to radicalize them.

realPtolemy 3y ago
They can already do that with some simple googling.
- borissk 3y ago
  No, they can't - they would have done it if they could. Producing a practical chemical weapon is a complicated task, with many steps that are not documented in publicly available sources.
  - acdha 3y ago
    That’s somewhat true – it’s not easy but not hard enough, as we saw with the Aum Shinrikyo attacks – but an LLM won’t magically have access to non-public instructions and, not having an understanding of the underlying principles, won’t be able to synthesize a safe process from public information.
    - pixl97 3y ago
      Eh that is up for debate. If I dump a library of chemistry books and industry books on chemistry and volatile chemicals its distinctly possible the model could generate this data.
      - acdha 3y ago
        Not without some kind of understanding of the underlying principles. If you were testing something verifiable in code you might be able to test candidates at scale, but this involves a number of real-world processes which would be hard to tackle that way.

dleeftink 3y ago

I wonder, how would this future differ from how big tech currently operates in relation to (F)OSS? Even with code/weights common to the public, a significant resource divide remains (e.g compute, infrastructure, R&D). I'm not arguing against more permissive licensing here, but I do not see it as a clear determinant for levelling the field either.

eigenket 3y ago

I don't understand how normal people having access to AI models helps you when big businesses are using them in unethical ways. Lets say for example I have access to exactly the models Facebook is using to target my elderly relatives with right-wing radicalising propaganda. How does that help me? This assumption that it helps somehow sounds like you've internalised some of the arguments people make about gun control and just assume those same points work in this case as well.

Baldbvrhunter 3y ago
> Lets say for example I have access to exactly the models Facebook is using to target my elderly relatives with right-wing radicalising propaganda. is it working?
visarga 3y ago
This small model could run locally and filter out bullshit/propaganda as configured by the user. Having control over the last model that filters your web is essential. Local models will be mandatory once the web gets filled with AI bots. You need your own AI bot to fight them off.
- eigenket 3y ago
  Most people don't even use ad blockers today. Hoping that people (especially the people who are vulnerable to such misinformation and actually need it) personally configure a propaganda filter AI is wildly optimistic.

potatoman22 3y ago

I think it's harmful to characterize "all" AI ethicists as a "priesthood" wanting to gatekeep access to these models. There are plenty of people who care both about the democratizing of these tools as well as safe and ethical use.

nullstyle 3y ago
Recommendations? Someone nonadjacent to the lesswrong smooth brains, pretty please.
- smoldesu 3y ago
  Seriously, I'd also really appreciate some examples of moderate AI ethicists. The vocal minority is all I've heard so far, and their arguments sound closer to fiction than science.
  - AndrewKemendo 3y ago
    I mean we wrote a whole ass engineering standard for this: https://standards.ieee.org/ieee/7010/7718/ https://standards.ieee.org/ieee/7010/7718/ I wrote the implementation annex
    - smoldesu 3y ago
      TIL. That's a neat standard and I'm glad it exists, it's an interesting reflection of what opt-in ethical frameworks can look like. For every reasonable and non-authoritarian suggestion I read for regulating AI, I feel like I wade through 10 Neuromancer-level takes. It's definitely a me-problem, I gotta stop scrolling through /new...
      - AndrewKemendo 3y ago
        Thanks. This was the effort of dozens of engineers, ethicists and systems people all done prior to the LLM revolution so it doesn’t have all the mystical junk that the newcomers seem to be latching onto.

jillesvangurp 3y ago

I think at this point, the cat is out of the bag. Relying on not so nice people complying with license legalese was never going to be a great way to impose control. All that does is stifle progress and innovation for those who are nice enough to abide by the law. But anyone with other intentions in say Russia, North Korea, China, etc. would not be constrained by such notions. Nor would criminal organizations, scam artists, etc. And there's a growing community of people doing work under proper OSS licenses where interesting things are happening at an accelerating pace. So, alternate licenses lack effectiveness, isolate you from that community and complicates collaboration, and they increasingly represent a minority of the overall research happening. Which makes these licenses a bit pointless. So, fixing this simplifies and normalizes things from a legal point of view which in turn simplifies commercialization, collaboration, and research. MS is being rational enough to recognize that there is value in that and is adjusting to this reality.

rmbyrro 3y ago
> anyone with other intentions in say Russia, North Korea, China, etc. I understand why the geopolitical focus, but let's not forget there are plenty of ill intentioned actors everywhere.
- NicoJuicy 3y ago
  Not with such bad intentions

acdha 3y ago

> What is interesting is that the AI “ethicists” all want to serve as a high priesthood controlling access to ML models in the name of safety. This is a very uncharitable take. I would suggest familiarizing yourself with the actual arguments rather than summaries on social media. There’s considerably more thought than you’re crediting them with, and extensive discussion around the risk you’re worried about along with proposed solutions which – unlike your “best defense” – could actually work.

visarga 3y ago
> if a non-signatory country is building a datacenter that might kill everyone on Earth, you should be willing to preemptively destroy that datacenter Eliezer Yudkowsky, high priest of AGI risk
- refulgentis 3y ago
  Yeah I think that's the posters point, "AI ethics" isn't "AGI risk", and I'll add A) Eliezer isn't a "high priest" he's just a guy, B) he plays a character and knows it. You'd be surprised how much you can advance in life just by avoiding talking or thinking about other people too much, much less grouping them. It's a fundamental part of our animal brains and it's gotten oh-so-much worse as we migrated to the internet. And it leads to epistemic closure. n.b. I think the AGI risk stuff is bunko and the original AI ethics cheerleaders ruined their own field. You don't need to agree to understand
Uehreka 3y ago
Moreover, in the next sentence GP confesses that they “think the biggest danger from AI is that these models will be used by those who control the models to control and censor what people are allowed to write”, revealing that they too harbor ethical concerns about AI, they’re just not one of “those” AI ethicists.
- cubefox 3y ago
  That's more a terminological accident I think. Those that describe themselves as working on "AI ethicis" in academia are mostly worried about stuff like AI's not saying something offensive or potentially discriminating related to race or sex, while people who use the term "AI risk" or "AI safety" are more worried about future risks like terrorism, war, or even human extinction. Thinking about it, both groups don't talk a lot about the risk from AI being used for censorship...

menacingly 3y ago

Every center of value develops a barnacle industry with their foot hovering over the brake pedal unless a tax is paid to their army of non-contributing people

eurekin 3y ago
Can you elaborate? :)

aleph_minus_one 3y ago

> What is interesting is that the AI “ethicists” all want to serve as a high priesthood controlling access to ML models in the name of safety. However, I think the biggest danger from AI is that these models will be used by those who control the models to control and censor what people are allowed to write. Who says that this is not an (or even the) actual hidden agenda behind these insane AI investments: building an infrastructure for large-scale censorship?