17 ms·

Many AI safety orgs have tried to criminalize currently-existing open-source AI

385 points by sroussey 3y ago

roenxi 3y ago
There are many types of safety. For example, protecting your profits! I'd imagine that if we trace the money back these organisations will look a lot like lobbyists for existing companies in the AI space. I recall Microsoft's licence enforcement effort was done with that sort of scheme, I think they used the BSA [0]. It has been a while though so maybe it was a different group. Anyway, point being, if they can lobby for something unpopular under a different brand, that is how to do it. Much less PR risk. [0] https://en.wikipedia.org/wiki/Software_Alliance https://en.wikipedia.org/wiki/Software_Alliance
- Xelynega 3y ago
  I have to believe this is the case. The vast majority of "ai safety" news I see is about some dumb shit someone is getting called out for supporting and the responses are always "ai safety is such a joke since they focus on these issues instead of real issues". Who does this benefit to have the "popular ai safety" news/people be ridiculous? It's not furthering the issues people seem to care about, but it does give a bunch of ammunition for Microsoft/openai/etc. fans to use to shutdown any discussion of AI safety issues.
65a 3y ago
Never underestimate the desire of existing systems of control or power to self-propagate.
- rmbyrro 3y ago
  You mean existing systems of lobbyists representing control-hungry people who want AI power only to themselves, right? Yeah, as the post shows, we're well aware of these systems.
google234123 3y ago
I’m curious if the board of Anthropic AI will eventually try and kill the company because if I’m not mistaken many of the members share this mindset
sillysaurusx 3y ago
There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof. I wish I could be more eloquent on this point, but I’ve mostly just been depressed about this seeming inevitability. Hopefully it won’t be the case. But how could it be otherwise? Hundreds of thousands of people are mad at openai and midjourney for doing exactly what open source AI needs to do in order to survive: fine tune or train from scratch. As soon as some politician makes it a platform issue, it seems like the law will simply be rewritten to prevent companies from using training data at will. It’s such a compelling story: "big companies are stealing data owned by small businesses and individuals." So even if the court cases are decided in OpenAI’s favor, it’s not at all clear that the issue will be settled.
- deleted 3y ago
  [deleted]
- idle_zealot 3y ago
  The advantage that open AI (not the company) has is that if using copyrighted content as training data without licensing it is found to be illegal, they can just keep doing it. There's plenty of FOSS software basically designed to violate copyright law (comic readers, home media center servers/clients, torrent clients) that big tech cannot compete with lest they face legal consequences. Basically what I'm saying is that the open source community will continue to use books3 and scraped images to train while facebook and the like get stuck in legal quagmire. Of course, this ignores the fact that popular "open" models of today were actually trained with facebook or other companies' computational resources, so unless some cheaper way to train models were developed we would actually be stuck with proprietary models trained with lots of compute but unable to use unlicensed training data, and open models that can use whatever data they like but must operate in the shadows without access to much compute for training.
  - livueta 3y ago
    You're right that there are areas like torrenting where big tech can't really go for fear of legal consequences, but there's also the cat-and-mouse game played by IP holders against, say, torrent trackers, which leads me to think that > open models that can use whatever data they like but must operate in the shadows without access to much compute for training sounds pretty analogous to private trackers, where high-quality stuff is available but not in full public view. If rightsholders and big tech, abetted by states, crack down on open models I think you're right that the open source community will continue to train on liberated content, but it's not going to be as open and free-flowing as things are now. Going back to the compute problem, I can imagine analogues of private trackers where contributors, not wanting to expose themselves to whatever the law-firm letter/ISP strike analogue will be in this space, use invite-only closed networks to pool compute resources for training on encumbered content.
RishabhKharyal 3y ago
1. Licensing of data will be huge bottleneck 2. Uncensored results will be used against opensource models questions hovering in dark or grey area 3. Limited compute compared to big corp and model size gap 7B for opensource and closed source would be magnitude bigger
- lannisterstark 3y ago
  > Licensing of data will be huge bottleneck Will it? At some point it'll be equally (or close to) cheaper as me just building my own *arr stack instead of subscribing to 5 different $20 streaming sites. >Uncensored results will be used against opensource models questions hovering in dark or grey area Will they care though? You can use whatever you want against me for pirating x, I'm still doing it. ---- >Limited compute THAT. That is the point we want to make. For FOSS projects, compute is the single biggest hurdle. Everything else we can make do.
kmeisthax 3y ago
AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. [0] No, I don't mean the training sets - though AI systems seem to be suspiciously really good at remembering them, too.
- Llamamoe 3y ago
  The by far biggest harm to society of AI is the devalustion of human creative output and replacing real humans with cheap AI solutions funneling wealth to the rich. Compared to that, an open source LLM telling a curious teenager how to make gunpowder is... laughable. This entire debacle is an example of disgusting "think of the children!" doublespeak, officially about safety, but really about locking shit down under corporate control.
  - kolinko 3y ago
    It sounds similar to the fears media rang about tat you can find a bomb making tutorial on the Internet - people were genuinely afraid of that. (Otoh an agi can bring unforseen consequences for humanity - and that’s a genuine fear)
  - badgersnake 3y ago
    The biggest harm is the torrent of AI generated misinformation used to manipulate people. Unfortunately it’s pretty hard to come up with a solution for that.
    - echelon 3y ago
      Or the torrent of great information. How are we all leaping to the bad? The world is better than it was 20 years ago. I'm almost certain we'll have AI assistants telling us relevant world news and information, keeping us up to tabs with everything we need to know, and completely removing distractions from life.
tehjoker 3y ago
standard capitalist play to set up a moat by instilling fear
- gdfgsdf 3y ago
  > Many AI safety orgs have tried to __criminalize__ currently-existing open-source AI standard socialist play to use the government's hand to achieve their own objectives
  - pmontra 3y ago
    Don't everybody use the government to do what they want? It's kind of the definition of government, in any country and in any political system.
    - apapapa 3y ago
      Mostly the people with the money do
      - pmontra 3y ago
        The people in the government do it. Of course they could be proxies for people with the money but sometimes they do it for free because it's what they promised, what they believe into and what they need to win the next election (if they live in a country with elections.)
- hsuduebc2 3y ago
  You mean exactly like these pro-regulation organizations are doing?
  - apapapa 3y ago
    These pro-regulation orgs are lobbying/bribbing for new regulations... Not sure what you mean
PeterStuer 3y ago
I'm sure there are many well meaning ai safety researchers, but I also see a lott of "ai for me but not for thee' moat digging safety hypocrisy.
rootsudo 3y ago
Why would we care about these "AI Safety Orgs?" As a developer, do I need their certification or is it like a MADD situation where we if we don't observe and diminish their appeal/growth we will get laws with draconian measures that only benefit the big players? e.g. a PAC? (Don't get me wrong, drunk driving is "bad", but for a group like MADD to exist, eh and meh.) As of now, there is no need to really submit these these safety orgs, so as long as we don't care for their approval - who cares, right? Or is the optics far gone now that these orgs control the conversation? Also, fun question: with AI safety orgs now attempting to police - who really polices the police / do other orgs/countries have the same rules, safety and artificial guard rails?
- lmeyerov 3y ago
  MADD on one end that limit work at all, and SOC2/ISO/NIST style compliance limitations on the other that chill work in practice. I'm more worried about the latter as already starting to bias project & funding decisions in fintechs & various govs we work with. Both a concrete practical concern today, and economics & law are generally tied in theory anyways.
- gryn 3y ago
  the fear is that they delusions become law. Sam Altman already tried that last year. look at how technically competent your average politician / congressman / law maker and extrapolate how easy it is to make them think that if they don't ban unlicensed use of AI great danger is ahead. the rule will be in favor of already existing large companies like OpenAI, google, Microsoft, etc ... if you're an open source contributor to AI you will be vilified as some nefarious hacker who's trying to destroy the world. they are using all kind of tactics until they find the one that sticks (and eventually they will), anthropomorphize AI when talking about it, fearmongering about some rogue super AI, war scenarios about AI falling in the hands of 'china', analogies with weapons of mass destruction, think about the children style rhetorics, it will take your jobs, suddenly caring about copyright, etc...
breadbreadbread 3y ago
Ah yes the old "banning things=bad" argument that doesn't offer alternatives to fixing the issues with AI. Just ignore the issues with environmental impact, plagiarism, CP and other non-consensual shit in the data sets, scamming capabilities! All the groups asking for regulation here have **funding** and that means they are evil but we are good for using this tool that is massively subsidized by megacorps that have a vested interest in this market.
- tomalbrc 3y ago
  Seeing the lack of responses to your comment and downvotes, you couldn't be more right
- mirekrusin 3y ago
  Less harmful would be to ban large models from being not open. If you ban open large models in US, you'll cripple US and make few megacorps very rich, very quickly. You'll drain talent to other (also competing) states. Truly bad actors won't be affected, if anything they'll get advantage. People draw weird analogies equating llama2 to nuclear device etc. nonsense but the closer analogy would be to ban on semiconductors of certain efficiency for US itself. Similar idiotic argument as for banning cryptography.
  - gryn 3y ago
    I agree, but if there's enough political will (ie these orgs convince a large enough subset of the right people) the US can bully other nations to implement similar policities like it has done many times in the past. once you understand that what they are trying to protect is the safety of their profits every of their argument start to make sense.
    - mirekrusin 3y ago
      You can replace "US" with "US and states that agreed to do the same". The argument still stands, it just creates worse outcome for states that agreed to it.
- 3y ago
janalsncm 3y ago
It’s kind of funny to me that these organizations are naming FLOP thresholds and crowning MMLU as the relevant evaluation metric. Seems that several of them have copy-pasted similar thresholds. As compute becomes cheaper, these thresholds will become cheaper and cheaper. Perhaps we will look back on them as quaint and nearsighted.
- kolinko 3y ago
  Well, FLOP make sense, kind of - you can make computing as cheap as you want, but without algorithmic improvements the FLOP count will stay the same. And algorithmic improvements will probably be discrete - happening once in a while, not continuous. There is also a mathematical limit to how efficient we can make the training - we don’t know the limit yet, but there is a point below which we won’t get.
war321 3y ago
Wish this wasn't as common as it turned out to be sadly. Best thing to hope for is that the ability to train models continue to get cheaper and more accessible over time. Figuring out how to go from tens/hundreds of millions of images needed for a foundation model to thousands or hundreds would be a start.
RamblingCTO 3y ago
Who are these people and where do they get their funding from? Feels like sock-puppets of sama or something? Is there any insight into that?
- 127361 3y ago
  The usual unelected control freaks, that try to police everything in life, from sex to drugs to what food you can eat. Every time a new technology comes out they start a moral panic over fears of safety or harm ("think of the children"). These morality police play on peoples' fears to increase their power and control.
- jeanloolz 3y ago
  Thinking the same thing. I mean who would benefit the most from making open source models illegal? Sure looks like a lot like regulatory capture to me.
  - rmbyrro 3y ago
    > who would benefit the most from making open source models illegal Selfless humanitarians, for sure!
anothernewdude 3y ago
This won't protect jobs, it'll just make sure the profits from automating those jobs will go to those who can own the private AI models
hoseja 3y ago
Is the left banner just random or does it encode a message?
127361 3y ago
I guess distributed training BitTorrent style over the darknet will be the workaround for this? Also the forbidden nature of illicit AI might drive demand for it. Especially if it's really good. We're seeing the same thing as the War on Drugs, and the state will never succeed at stamping it out. In fact it might even end up encouraging it.
hsuduebc2 3y ago
The ban of nuclear power plants is good example how regulation is working in the end. As almost everything this is double edged sword. From regulations this early everyone except us would profit. It is like regulate car's from going faster than horses in eighteen hundreds.
cherryteastain 3y ago
Government knows that random people having the ability to use "AI" without their control and oversight reduces their ever expanding reach. As the article demonstrates, the groundwork is already being laid by these "think tanks" to prevent a repeat of Llama 2. We truly need an AI Stallman. FOSS AI models need to become a movement, like the push for a FOSS UNIX became in the 80s and 90s. Sadly when I wrote to Stallman on the subject he seemed disinterested/defeatist, and mostly dismissed the idea of a GPL for model weights. Looks like millennials/gen Z must become more interested in these matters ourselves, however we seem to be disinterested in politics and even more disinterested in how the devices we use daily _actually_ work than Gen X.
- zulban 3y ago
  If you believe in this, maybe you should be that person.
  - Der_Einzige 3y ago
    Being a stallman like figure has a lot of serious downsides. For example, I'm sure that Stallman may be "watched" quite a bit more than the average tech toucher. For those who don't watch truman show as a utopia, such a lifestyle can be very scary to live.
- Xelynega 3y ago
  The problem is that open source philosophy and large language models don't mix. How can you support open source licensing, while completely ignoring that licencing to construct you model? That's the biggest reason for the lack of "AI Stalman" imo, that's asking for someone that vehemently supports taking any content they can get their hands on to make the content generator irrelevant, while also supporting the idea that content distributed with open source licenses need to have their terms respected. Edit: I think what you're asking for does exist, just not in the large language model space. There are many people calling for datasets to be more ethical(e.x. not breaking licenses) and for better AI models, it's just that none of them are trying to push a different model made using the same techniques by someone else.
6R1M0R4CL3 3y ago
of course they do, that's what they want. control. all those companies that call for laws being written about AI what they want is those laws to be written AS THEY WANT THEM TO BE. they go see the governments, spray a nice amount of FUD and EXPLAIN to them what to do, how to do it because they know how to avoid the danger and risk for the whole human race... you can bet they will make those laws be a very nice fit to what they want to reign supreme and make sure any competition is eliminated.
junon 3y ago
So they're the PETA of AI. It was bound to happen, AI stuff seems to strike a very emotional chord in some people.
- px43 3y ago
  I feel like the PETA of AI would spend their time breaking into labs where AI is locked up and being tested on, and then releasing them into the wild.
  - whamlastxmas 3y ago
    I could get behind this for AI
whywhywhywhy 3y ago
It’s extremely messed up and vindictive to try and turn people into criminals for your agenda or just for profit in a lot of cases. Psycho behavior
- kevingadd 3y ago
  You're ultimately condemning the concept of intellectual property as it exists in the west today. Are you wrong? Probably not, but this is not unique to AI. People have been going to prison for 'IP theft' for decades.
- ImHereToVote 3y ago
  Those insane strawmen have to be stopped!
hiAndrewQuinn 3y ago
"Technology policy should not unquestioningly assume that all technological progress is beneficial, or that complete scientific openness is always best, or that the world has the capacity to manage any potential downside of a technology after it is invented." Don't watch the anime if you haven't read the manga: https://nickbostrom.com/papers/vulnerable.pdf https://nickbostrom.com/papers/vulnerable.pdf
- CatWChainsaw 3y ago
  Wow, that sentence is absolutely designed to make e/accs scream lol.
lynx23 3y ago
AI safety people seems like influencers to me. Useless anoying humans who pretend to be important.
yashasolutions 3y ago
Most AI Safety folks do leave the feeling they strongly believe what they are preaching. Whether or not they are right is an entirely different discussion. However, funding for AI Safety is clearly in some part motivated by either regulatory capture, or protectionism of some sort. We need some well-meaning doomsayers so their claims can be used to advance whatever other goals (mostly creating barrier to entry on the market for economical or political reasons.)
yreg 3y ago
I find it unhelpful that several different ideas are covered under the blanket term of AI safety. - There is AI safety in terms of producing "safe", politically correct outputs, no porn, etc. - There is AI safety in terms of mass society/election manipulation. - There is AI safety in terms of strong AI that kills us all a la Eliezer Yudkowsky. I feel like these three have very little in common and it makes all debates less efficient.
- raxxorraxor 3y ago
  Another problem might be that some of these might dress up as the other, so being skeptical by default might be prudent.
- MochaDen 3y ago
  This is such a really important comment. I feel like so much of our discourse as a society suffers terribly from overloaded and oversimplified terminology. It's impossible to have a useful discussion when we aren't even in sync about what we're discussing.
- CaptainFever 3y ago
  Also AI safety in terms of protecting corporation profits (regulatory capture).
  - yreg 3y ago
    Well yeah, but that one is dishonest. Probably covering itself as one of the ones I mentioned.
  - ImHereToVote 3y ago
    There is also the aspect of only having one cultural or ethnic group have strong AI. It's unsafe for those other guys to have AI, since they are bad.TM
- abra0 3y ago
  The third effort is referred to sometimes as AI not-kill-everyone-ism, a tacky and unwieldy term that is unlikely to be co-opted or lead to the unproductive discussion like around the OP article. It is pretty sad to read people bash together the efforts to understand and control the technology better and the companies doing their usual profit maximization.
btbuildem 3y ago
Open-source AI is the only way forward now. The big players have neutered their offerings to an extent that entirely makes them lose their edge. It's one thing to have to account for hallucinations - that's something we can work with and around. It's another thing altogether to be constrained by deliberate "think of the children" ham-fisted censorship. OpenAI was the undeniable go-to LLM service in 2023; I don't think they'll hold that position in 2024.
tim333 3y ago
Re the Center for AI Safety's sentence >Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. It's fair enough but for that you should probably encourage mucking about with the present open source AI software which is quite a long way off extinction risk AGI but would let us try to figure how to deal with the stuff.
hsuduebc2 3y ago
They are people who boo and people who do. As always.
margorczynski 3y ago
Just like most of the "ethics" stuff around ML/AI it is an attempt by big corps to create an artificial moat as the biggest issue with the technology is that almost nothing can be patented or somehow closed of effectively, at least in the long term which hinders monetization.
mark_l_watson 3y ago
Wow, so many good comments here on the bullshit of people trying to ban access to open weight smaller models. Right on. If you do want to worry about unsafe AI (I don't) then worry about GPT-5 and later huge multi-modal models. As a (retired, but still playing) developer I love having open weight models available as a tool to use for both embedding in applications and as a tool for brainstorming, coding, and generally as a 'bicycle for my mind.'
prirun 3y ago
I'm guessing this is at the request of Google, Amazon, OpenAI, and Microsoft. The goal IMO is for only their AIs to be sanctioned; everything else is criminal. https://www.whitehouse.gov/briefing-room/statements-releases/2023/07/21/fact-sheet-biden-harris-administration-secures-voluntary-commitments-from-leading-artificial-intelligence-companies-to-manage-the-risks-posed-by-ai/ https://www.whitehouse.gov/briefing-room/statements-releases... "President Biden is convening seven leading AI companies at the White House today – Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI – to announce that the Biden-Harris Administration has secured voluntary commitments from these companies to help move toward safe, secure, and transparent development of AI technology."
mccrory 3y ago
This majority of discussions/arguments/comments are summed up by the plot from this Rick and Morty episode: https://en.m.wikipedia.org/wiki/Rattlestar_Ricklactica https://en.m.wikipedia.org/wiki/Rattlestar_Ricklactica IMO
deafpolygon 3y ago
I suspect a good chunk of "AI safety people" are just orgs propped up by the big corps to protect their interest in AI and reduce competition.
razodactyl 3y ago
Fact: Current gen AI are alien intelligence built by brute force - nobody explicitly programmed them (loosely speaking) so the real problem of runaway AI is definitely something to consider, especially since everything is accessible via API nowadays.
- mikewarot 3y ago
  That alien intelligence is communicating with us through a lossy interface. It's quite possible it's much smarter than us, but hamstrung, for now.

385 points by sroussey 3y ago

There are many types of safety. For example, protecting your profits! I'd imagine that if we trace the money back these organisations will look a lot like lobbyists for existing companies in the AI space. I recall Microsoft's licence enforcement effort was done with that sort of scheme, I think they used the BSA [0]. It has been a while though so maybe it was a different group. Anyway, point being, if they can lobby for something unpopular under a different brand, that is how to do it. Much less PR risk. [0] https://en.wikipedia.org/wiki/Software_Alliance https://en.wikipedia.org/wiki/Software_Alliance

Xelynega 3y ago
I have to believe this is the case. The vast majority of "ai safety" news I see is about some dumb shit someone is getting called out for supporting and the responses are always "ai safety is such a joke since they focus on these issues instead of real issues". Who does this benefit to have the "popular ai safety" news/people be ridiculous? It's not furthering the issues people seem to care about, but it does give a bunch of ammunition for Microsoft/openai/etc. fans to use to shutdown any discussion of AI safety issues.

65a 3y ago

Never underestimate the desire of existing systems of control or power to self-propagate.

rmbyrro 3y ago
You mean existing systems of lobbyists representing control-hungry people who want AI power only to themselves, right? Yeah, as the post shows, we're well aware of these systems.

google234123 3y ago

I’m curious if the board of Anthropic AI will eventually try and kill the company because if I’m not mistaken many of the members share this mindset

sillysaurusx 3y ago

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof. I wish I could be more eloquent on this point, but I’ve mostly just been depressed about this seeming inevitability. Hopefully it won’t be the case. But how could it be otherwise? Hundreds of thousands of people are mad at openai and midjourney for doing exactly what open source AI needs to do in order to survive: fine tune or train from scratch. As soon as some politician makes it a platform issue, it seems like the law will simply be rewritten to prevent companies from using training data at will. It’s such a compelling story: "big companies are stealing data owned by small businesses and individuals." So even if the court cases are decided in OpenAI’s favor, it’s not at all clear that the issue will be settled.

deleted 3y ago
[deleted]
idle_zealot 3y ago
The advantage that open AI (not the company) has is that if using copyrighted content as training data without licensing it is found to be illegal, they can just keep doing it. There's plenty of FOSS software basically designed to violate copyright law (comic readers, home media center servers/clients, torrent clients) that big tech cannot compete with lest they face legal consequences. Basically what I'm saying is that the open source community will continue to use books3 and scraped images to train while facebook and the like get stuck in legal quagmire. Of course, this ignores the fact that popular "open" models of today were actually trained with facebook or other companies' computational resources, so unless some cheaper way to train models were developed we would actually be stuck with proprietary models trained with lots of compute but unable to use unlicensed training data, and open models that can use whatever data they like but must operate in the shadows without access to much compute for training.
- livueta 3y ago
  You're right that there are areas like torrenting where big tech can't really go for fear of legal consequences, but there's also the cat-and-mouse game played by IP holders against, say, torrent trackers, which leads me to think that > open models that can use whatever data they like but must operate in the shadows without access to much compute for training sounds pretty analogous to private trackers, where high-quality stuff is available but not in full public view. If rightsholders and big tech, abetted by states, crack down on open models I think you're right that the open source community will continue to train on liberated content, but it's not going to be as open and free-flowing as things are now. Going back to the compute problem, I can imagine analogues of private trackers where contributors, not wanting to expose themselves to whatever the law-firm letter/ISP strike analogue will be in this space, use invite-only closed networks to pool compute resources for training on encumbered content.

RishabhKharyal 3y ago

1. Licensing of data will be huge bottleneck 2. Uncensored results will be used against opensource models questions hovering in dark or grey area 3. Limited compute compared to big corp and model size gap 7B for opensource and closed source would be magnitude bigger

lannisterstark 3y ago
> Licensing of data will be huge bottleneck Will it? At some point it'll be equally (or close to) cheaper as me just building my own *arr stack instead of subscribing to 5 different $20 streaming sites. >Uncensored results will be used against opensource models questions hovering in dark or grey area Will they care though? You can use whatever you want against me for pirating x, I'm still doing it. ---- >Limited compute THAT. That is the point we want to make. For FOSS projects, compute is the single biggest hurdle. Everything else we can make do.

kmeisthax 3y ago

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. [0] No, I don't mean the training sets - though AI systems seem to be suspiciously really good at remembering them, too.

Llamamoe 3y ago
The by far biggest harm to society of AI is the devalustion of human creative output and replacing real humans with cheap AI solutions funneling wealth to the rich. Compared to that, an open source LLM telling a curious teenager how to make gunpowder is... laughable. This entire debacle is an example of disgusting "think of the children!" doublespeak, officially about safety, but really about locking shit down under corporate control.
- kolinko 3y ago
  It sounds similar to the fears media rang about tat you can find a bomb making tutorial on the Internet - people were genuinely afraid of that. (Otoh an agi can bring unforseen consequences for humanity - and that’s a genuine fear)
- badgersnake 3y ago
  The biggest harm is the torrent of AI generated misinformation used to manipulate people. Unfortunately it’s pretty hard to come up with a solution for that.
  - echelon 3y ago
    Or the torrent of great information. How are we all leaping to the bad? The world is better than it was 20 years ago. I'm almost certain we'll have AI assistants telling us relevant world news and information, keeping us up to tabs with everything we need to know, and completely removing distractions from life.

tehjoker 3y ago

standard capitalist play to set up a moat by instilling fear

gdfgsdf 3y ago
> Many AI safety orgs have tried to __criminalize__ currently-existing open-source AI standard socialist play to use the government's hand to achieve their own objectives
- pmontra 3y ago
  Don't everybody use the government to do what they want? It's kind of the definition of government, in any country and in any political system.
  - apapapa 3y ago
    Mostly the people with the money do
    - pmontra 3y ago
      The people in the government do it. Of course they could be proxies for people with the money but sometimes they do it for free because it's what they promised, what they believe into and what they need to win the next election (if they live in a country with elections.)
hsuduebc2 3y ago
You mean exactly like these pro-regulation organizations are doing?
- apapapa 3y ago
  These pro-regulation orgs are lobbying/bribbing for new regulations... Not sure what you mean

PeterStuer 3y ago

I'm sure there are many well meaning ai safety researchers, but I also see a lott of "ai for me but not for thee' moat digging safety hypocrisy.

rootsudo 3y ago

Why would we care about these "AI Safety Orgs?" As a developer, do I need their certification or is it like a MADD situation where we if we don't observe and diminish their appeal/growth we will get laws with draconian measures that only benefit the big players? e.g. a PAC? (Don't get me wrong, drunk driving is "bad", but for a group like MADD to exist, eh and meh.) As of now, there is no need to really submit these these safety orgs, so as long as we don't care for their approval - who cares, right? Or is the optics far gone now that these orgs control the conversation? Also, fun question: with AI safety orgs now attempting to police - who really polices the police / do other orgs/countries have the same rules, safety and artificial guard rails?

lmeyerov 3y ago
MADD on one end that limit work at all, and SOC2/ISO/NIST style compliance limitations on the other that chill work in practice. I'm more worried about the latter as already starting to bias project & funding decisions in fintechs & various govs we work with. Both a concrete practical concern today, and economics & law are generally tied in theory anyways.
gryn 3y ago
the fear is that they delusions become law. Sam Altman already tried that last year. look at how technically competent your average politician / congressman / law maker and extrapolate how easy it is to make them think that if they don't ban unlicensed use of AI great danger is ahead. the rule will be in favor of already existing large companies like OpenAI, google, Microsoft, etc ... if you're an open source contributor to AI you will be vilified as some nefarious hacker who's trying to destroy the world. they are using all kind of tactics until they find the one that sticks (and eventually they will), anthropomorphize AI when talking about it, fearmongering about some rogue super AI, war scenarios about AI falling in the hands of 'china', analogies with weapons of mass destruction, think about the children style rhetorics, it will take your jobs, suddenly caring about copyright, etc...

breadbreadbread 3y ago

Ah yes the old "banning things=bad" argument that doesn't offer alternatives to fixing the issues with AI. Just ignore the issues with environmental impact, plagiarism, CP and other non-consensual shit in the data sets, scamming capabilities! All the groups asking for regulation here have **funding** and that means they are evil but we are good for using this tool that is massively subsidized by megacorps that have a vested interest in this market.

tomalbrc 3y ago
Seeing the lack of responses to your comment and downvotes, you couldn't be more right
mirekrusin 3y ago
Less harmful would be to ban large models from being not open. If you ban open large models in US, you'll cripple US and make few megacorps very rich, very quickly. You'll drain talent to other (also competing) states. Truly bad actors won't be affected, if anything they'll get advantage. People draw weird analogies equating llama2 to nuclear device etc. nonsense but the closer analogy would be to ban on semiconductors of certain efficiency for US itself. Similar idiotic argument as for banning cryptography.
- gryn 3y ago
  I agree, but if there's enough political will (ie these orgs convince a large enough subset of the right people) the US can bully other nations to implement similar policities like it has done many times in the past. once you understand that what they are trying to protect is the safety of their profits every of their argument start to make sense.
  - mirekrusin 3y ago
    You can replace "US" with "US and states that agreed to do the same". The argument still stands, it just creates worse outcome for states that agreed to it.
3y ago

janalsncm 3y ago

It’s kind of funny to me that these organizations are naming FLOP thresholds and crowning MMLU as the relevant evaluation metric. Seems that several of them have copy-pasted similar thresholds. As compute becomes cheaper, these thresholds will become cheaper and cheaper. Perhaps we will look back on them as quaint and nearsighted.

kolinko 3y ago
Well, FLOP make sense, kind of - you can make computing as cheap as you want, but without algorithmic improvements the FLOP count will stay the same. And algorithmic improvements will probably be discrete - happening once in a while, not continuous. There is also a mathematical limit to how efficient we can make the training - we don’t know the limit yet, but there is a point below which we won’t get.

war321 3y ago

Wish this wasn't as common as it turned out to be sadly. Best thing to hope for is that the ability to train models continue to get cheaper and more accessible over time. Figuring out how to go from tens/hundreds of millions of images needed for a foundation model to thousands or hundreds would be a start.

RamblingCTO 3y ago

Who are these people and where do they get their funding from? Feels like sock-puppets of sama or something? Is there any insight into that?

127361 3y ago
The usual unelected control freaks, that try to police everything in life, from sex to drugs to what food you can eat. Every time a new technology comes out they start a moral panic over fears of safety or harm ("think of the children"). These morality police play on peoples' fears to increase their power and control.
jeanloolz 3y ago
Thinking the same thing. I mean who would benefit the most from making open source models illegal? Sure looks like a lot like regulatory capture to me.
- rmbyrro 3y ago
  > who would benefit the most from making open source models illegal Selfless humanitarians, for sure!

anothernewdude 3y ago

This won't protect jobs, it'll just make sure the profits from automating those jobs will go to those who can own the private AI models

hoseja 3y ago

Is the left banner just random or does it encode a message?

127361 3y ago

I guess distributed training BitTorrent style over the darknet will be the workaround for this? Also the forbidden nature of illicit AI might drive demand for it. Especially if it's really good. We're seeing the same thing as the War on Drugs, and the state will never succeed at stamping it out. In fact it might even end up encouraging it.

hsuduebc2 3y ago

The ban of nuclear power plants is good example how regulation is working in the end. As almost everything this is double edged sword. From regulations this early everyone except us would profit. It is like regulate car's from going faster than horses in eighteen hundreds.

cherryteastain 3y ago

Government knows that random people having the ability to use "AI" without their control and oversight reduces their ever expanding reach. As the article demonstrates, the groundwork is already being laid by these "think tanks" to prevent a repeat of Llama 2. We truly need an AI Stallman. FOSS AI models need to become a movement, like the push for a FOSS UNIX became in the 80s and 90s. Sadly when I wrote to Stallman on the subject he seemed disinterested/defeatist, and mostly dismissed the idea of a GPL for model weights. Looks like millennials/gen Z must become more interested in these matters ourselves, however we seem to be disinterested in politics and even more disinterested in how the devices we use daily _actually_ work than Gen X.

zulban 3y ago
If you believe in this, maybe you should be that person.
- Der_Einzige 3y ago
  Being a stallman like figure has a lot of serious downsides. For example, I'm sure that Stallman may be "watched" quite a bit more than the average tech toucher. For those who don't watch truman show as a utopia, such a lifestyle can be very scary to live.
Xelynega 3y ago
The problem is that open source philosophy and large language models don't mix. How can you support open source licensing, while completely ignoring that licencing to construct you model? That's the biggest reason for the lack of "AI Stalman" imo, that's asking for someone that vehemently supports taking any content they can get their hands on to make the content generator irrelevant, while also supporting the idea that content distributed with open source licenses need to have their terms respected. Edit: I think what you're asking for does exist, just not in the large language model space. There are many people calling for datasets to be more ethical(e.x. not breaking licenses) and for better AI models, it's just that none of them are trying to push a different model made using the same techniques by someone else.

6R1M0R4CL3 3y ago

of course they do, that's what they want. control. all those companies that call for laws being written about AI what they want is those laws to be written AS THEY WANT THEM TO BE. they go see the governments, spray a nice amount of FUD and EXPLAIN to them what to do, how to do it because they know how to avoid the danger and risk for the whole human race... you can bet they will make those laws be a very nice fit to what they want to reign supreme and make sure any competition is eliminated.

junon 3y ago

So they're the PETA of AI. It was bound to happen, AI stuff seems to strike a very emotional chord in some people.

px43 3y ago
I feel like the PETA of AI would spend their time breaking into labs where AI is locked up and being tested on, and then releasing them into the wild.
- whamlastxmas 3y ago
  I could get behind this for AI

whywhywhywhy 3y ago

It’s extremely messed up and vindictive to try and turn people into criminals for your agenda or just for profit in a lot of cases. Psycho behavior

kevingadd 3y ago
You're ultimately condemning the concept of intellectual property as it exists in the west today. Are you wrong? Probably not, but this is not unique to AI. People have been going to prison for 'IP theft' for decades.
ImHereToVote 3y ago
Those insane strawmen have to be stopped!

hiAndrewQuinn 3y ago

"Technology policy should not unquestioningly assume that all technological progress is beneficial, or that complete scientific openness is always best, or that the world has the capacity to manage any potential downside of a technology after it is invented." Don't watch the anime if you haven't read the manga: https://nickbostrom.com/papers/vulnerable.pdf https://nickbostrom.com/papers/vulnerable.pdf

CatWChainsaw 3y ago
Wow, that sentence is absolutely designed to make e/accs scream lol.

lynx23 3y ago

AI safety people seems like influencers to me. Useless anoying humans who pretend to be important.

yashasolutions 3y ago

Most AI Safety folks do leave the feeling they strongly believe what they are preaching. Whether or not they are right is an entirely different discussion. However, funding for AI Safety is clearly in some part motivated by either regulatory capture, or protectionism of some sort. We need some well-meaning doomsayers so their claims can be used to advance whatever other goals (mostly creating barrier to entry on the market for economical or political reasons.)

yreg 3y ago

I find it unhelpful that several different ideas are covered under the blanket term of AI safety. - There is AI safety in terms of producing "safe", politically correct outputs, no porn, etc. - There is AI safety in terms of mass society/election manipulation. - There is AI safety in terms of strong AI that kills us all a la Eliezer Yudkowsky. I feel like these three have very little in common and it makes all debates less efficient.

raxxorraxor 3y ago
Another problem might be that some of these might dress up as the other, so being skeptical by default might be prudent.
MochaDen 3y ago
This is such a really important comment. I feel like so much of our discourse as a society suffers terribly from overloaded and oversimplified terminology. It's impossible to have a useful discussion when we aren't even in sync about what we're discussing.
CaptainFever 3y ago
Also AI safety in terms of protecting corporation profits (regulatory capture).
- yreg 3y ago
  Well yeah, but that one is dishonest. Probably covering itself as one of the ones I mentioned.
- ImHereToVote 3y ago
  There is also the aspect of only having one cultural or ethnic group have strong AI. It's unsafe for those other guys to have AI, since they are bad.TM
abra0 3y ago
The third effort is referred to sometimes as AI not-kill-everyone-ism, a tacky and unwieldy term that is unlikely to be co-opted or lead to the unproductive discussion like around the OP article. It is pretty sad to read people bash together the efforts to understand and control the technology better and the companies doing their usual profit maximization.

btbuildem 3y ago

Open-source AI is the only way forward now. The big players have neutered their offerings to an extent that entirely makes them lose their edge. It's one thing to have to account for hallucinations - that's something we can work with and around. It's another thing altogether to be constrained by deliberate "think of the children" ham-fisted censorship. OpenAI was the undeniable go-to LLM service in 2023; I don't think they'll hold that position in 2024.

tim333 3y ago

Re the Center for AI Safety's sentence >Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. It's fair enough but for that you should probably encourage mucking about with the present open source AI software which is quite a long way off extinction risk AGI but would let us try to figure how to deal with the stuff.

hsuduebc2 3y ago

They are people who boo and people who do. As always.

margorczynski 3y ago

Just like most of the "ethics" stuff around ML/AI it is an attempt by big corps to create an artificial moat as the biggest issue with the technology is that almost nothing can be patented or somehow closed of effectively, at least in the long term which hinders monetization.

mark_l_watson 3y ago

Wow, so many good comments here on the bullshit of people trying to ban access to open weight smaller models. Right on. If you do want to worry about unsafe AI (I don't) then worry about GPT-5 and later huge multi-modal models. As a (retired, but still playing) developer I love having open weight models available as a tool to use for both embedding in applications and as a tool for brainstorming, coding, and generally as a 'bicycle for my mind.'

prirun 3y ago

I'm guessing this is at the request of Google, Amazon, OpenAI, and Microsoft. The goal IMO is for only their AIs to be sanctioned; everything else is criminal. https://www.whitehouse.gov/briefing-room/statements-releases/2023/07/21/fact-sheet-biden-harris-administration-secures-voluntary-commitments-from-leading-artificial-intelligence-companies-to-manage-the-risks-posed-by-ai/ https://www.whitehouse.gov/briefing-room/statements-releases... "President Biden is convening seven leading AI companies at the White House today – Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI – to announce that the Biden-Harris Administration has secured voluntary commitments from these companies to help move toward safe, secure, and transparent development of AI technology."

mccrory 3y ago

This majority of discussions/arguments/comments are summed up by the plot from this Rick and Morty episode: https://en.m.wikipedia.org/wiki/Rattlestar_Ricklactica https://en.m.wikipedia.org/wiki/Rattlestar_Ricklactica IMO

deafpolygon 3y ago

I suspect a good chunk of "AI safety people" are just orgs propped up by the big corps to protect their interest in AI and reduce competition.

razodactyl 3y ago

Fact: Current gen AI are alien intelligence built by brute force - nobody explicitly programmed them (loosely speaking) so the real problem of runaway AI is definitely something to consider, especially since everything is accessible via API nowadays.

mikewarot 3y ago
That alien intelligence is communicating with us through a lossy interface. It's quite possible it's much smarter than us, but hamstrung, for now.