3 ms·
People have bad memories. I keep going back to the actual announcement because what they actually say is: """This decision, as well as our discussion of it, is
by ben_w 3y ago
People have bad memories. I keep going back to the actual announcement because what they actually say is:
"""This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. Other disciplines such as biotechnology and cybersecurity have long had active debates about responsible publication in cases with clear misuse potential, and we hope that our experiment will serve as a case study for more nuanced discussions of model and code release decisions in the AI community.
We are aware that some researchers have the technical capacity to reproduce and open source our results. We believe our release strategy limits the initial set of organizations who may choose to do this, and gives the AI community more time to have a discussion about the implications of such systems."""
- https://openai.com/index/better-language-models/ https://openai.com/index/better-language-models/
> The only remotely possible "safety" part I would acknowledge is that it should be balanced against biases if used in systems like loans, grants, etc.
That's a very mid-1990s view of algorithmic risk, given models like this are already being used for scams and propaganda.
- LegionMammal978 3y agoI'd imagine there's a wide spectrum between "release the latest model immediately to everyone with no idea what it's capable of" and OpenAI's apparent "release the model (or increasingly, any information about it) literally never, not even when it's long been left in the dust".
- ben_w 3y agoYes, indeed. However, given the capacity for some of the more capable downloadable models to enable automation of fraud, I am not convinced OpenAI is incorrect here. If OpenAI and Facebook both get sued out of existence due to their models being used for fraud and them being deemed liable for that fraud, the OpenAI models become unavailable, the Facebook models remain circulating forever
- modeless 3y ago> we hope that our experiment will serve as a case study for more nuanced discussions People trot this out every time this comes up, but this actually makes it even worse. This was only part of the reason, the other part was that they seemed to legitimately think there could be a real reason to withhold the model ("we are not sure"). In hindsight this looks silly, and I don't believe it improved the "discussion" in any way. If anything it seems to give ammunition to the people who say the concerns are overblown and self-serving, which I'm sure is not what OpenAI intended. So to me this is a failure on both counts, and this was foreseeable at the time.
- somenameforme 3y agoHere is the more relevant paper released by OpenAI. [1] It obsesses on dangers, misuse, and abuse for a model which was mostly incoherent. [1] - https://arxiv.org/pdf/1908.09203 https://arxiv.org/pdf/1908.09203
- ben_w 2y agoGiven that "being 100% right" isn't possible because they're not omniscient, would you rather they embarrass themselves by being a little too cautious and saying "let's experiment" and "here's what we found from our experiment" 1.5 versions before it turned out to matter, or would you instead prefer they had let everyone download 3.5 without any restrictions because they hadn't even stopped to think about what might possibly go wrong? (Failing to stop and think in advance what might go wrong, seems to be why Facebook didn't invest much in moderating the content in Burma until much too late; they'd have also looked to many people like they were 'worrying about nothing' if they'd done that investing before the genocide gave everyone a concrete example of why that matters) All of the stuff I saw OpenAI being concerned about in the link, has in fact happened with the bigger models (that said, this is a skim-reading, not an exhaustive analysis): -- """We chose a staged release process, releasing the smallest model in February, but withholding larger models due to concerns about the potential for misuse, such as generating fake news content, impersonating others in email, or au- tomating abusive social media content production [56].""" (Basically all of that) """One key variable affecting the social impact of language models is the extent to which humans and machines can detect outputs. We found reasons for optimism as well as reasons to continue being vigilant about the misuse of language models going forward. Our thoughts on detection at this time are: • Humans can be deceived by text generated by GPT-2 and other successful language models, and human detectability will likely become increasingly more difficult.""" (I've been accused, here, of being an LLM. Have you, yet?) """Security: There is a tradeoff between the number of partners and the likelihood of a model being prematurely released, accounting for hacks and leaks. Fairness: The high cost of compute used in powerful models like GPT-2 raises concerns about accessibility and equity in future AI research [13]. Private model sharing should not excessively harm researchers with limited computing resources, and conflicts of interest related to model sharing should be avoided in commercial contexts.""" (One of the GPT-2 models got leaked, so did one of the LLaMA models; at the same time, there's loads of people who think it's big and clever to call OpenAI "ClosedAI" and even to throw hissy fits about the other models failing to provide the training set and thus revealing that they have no idea how expensive the models are to train).
- lannisterstark 3y agoIf you're including actual announcement then why ignore this portion too? > Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model(opens in a new window) for researchers to experiment with, as well as a technical paper(opens in a new window). If you note, that's pretty much verbatim to what I said. So no, people don't have defective memories, some people just selectively quote stuff :P You should actually read the paper associated with it. It's largely a journey in "why would you think that" reading.
- ben_w 3y ago> If you're including actual announcement then why ignore this portion too? Because: > some people just selectively quote stuff And that's what I'm demonstrating with the bit I did quote, which substantially changes the frame of what you're saying. Our written language doesn't allow us to put all the caveats and justifications into the same space, and therefore it is an error to ignore a later section of the same document that makes the what and why clear, along with caveating this as "an experiment" and "we know others can do this" and "we're not sure if we're right".
- IanCal 3y ago"so advanced it's a threat to humankind" and "some people might use this in a bad way" are incredibly different.