22 ms·

OpenAI should now change their name to ClosedAI

503 points by gitgud 6y ago

timkam 6y ago
Can anyone point to a more nuanced perspective on this? Ideally not only with regards to OpenAI; it is still quite common that research published in some of the "top" venues does not disclose the underlying source code. Anecdote: I once pushed researchers to publish their code when I was peer-reviewing and it turned out that the code was super slow in comparison to the benchmark algorithms they compared with (comparison only included accuracy etc, not speed), something that I am sure the authors were aware of, but chose to not report in the paper.
- unityByFreedom 6y ago
  > it is still quite common that research published in some of the "top" venues does not disclose the underlying source code. The whole point of "OpenAI" was to share their innovations openly. They were a non profit. Now they're a "capped profit" and not sharing what's needed to reproduce their results (full code and training data). They acquired talent on a bunch of good will and shifted into a for profit.
- IAmEveryone 6y ago
  I don't necessarily agree with the following with regards to OpenAI, but it is what I have been told by a few professors and other academics in bioinformatics: Money is terribly tight in academia, and even at hot think tanks like OpenAI. The difference between a working but not very pretty prototype and a polished product you are willing to be associated with may seem like just a bunch of trivial boilerplate such as documentation or tests. It is indeed boring work (at least for scientists), but a lot of it. And if the intention is to publish and support something on an ongoing basis, and for widespread use, you probably need to invest 5x to 10x as much time and money. So in the end most code in academia is hidden because it's hideous, not to hide how they hardcoded all the good jokes GTP-3 comes up with.
  - woah 6y ago
    Such an odd attitude. There’s no way it takes 5-10x engineering effort to write some tests for components of a one-shot python script, as most scientific code is. It’s maybe the easiest type of software to test. Would this type of attitude fly for anything else? Can you imagine if chemists worked like this? “Yea most chemists don’t like to release their procedures, since taking the trouble of accurately measuring out reagents could be 5-10x the effort and grant money is tight. Plus sometimes they don’t want to go all the way to the cabinet to grab a beaker so they just do the reaction in a coffee mug. The truth is that sometimes the procedure is just too ugly to release. And if they come out of the lab with a new chemical, what does it matter?”
    - mennis16 6y ago
      I mean a lot of biology papers are loose on protocol details to be honest, dunno about Chemistry but for Bio it can often be difficult to exactly replicate a method because of lack of details alone. That said I think this should change given how easy it is these days to share supplemental info online. As far as ugly code I wouldn't be surprised if a lot of academics won't take the time to fix it up, but that doesn't mean it shouldn't be released IMO. Maybe an open source community would pop up that could help clean this code for them.
- albntomat0 6y ago
  Here's my alternative view: If OpenAI had open sourced GPT-3, there would be an equivalently angry thread about how they were endangering democracy/social order/etc, and not being responsible with the powerful tool they had created (see other threads on HN regarding those working in facial recognition). Both that group, and the one posting here have valid points, and there would be strong, valid critique of their decision no matter which way they chose.
  - sudosysgen 6y ago
    But we do realize that sooner or later a model even more powerful will be released to the public, right? And as far as protecting democracy, I assure you that the geopolitical enemies of your nation either have similar models, or will have them very soon. There is very significant investment on ML models for text manipulation going on behind closed doors, funded by States.
    - albntomat0 6y ago
      > But we do realize that sooner or later a model even more powerful will be released to the public, right? And at that time, we can equally criticize whomever releases it. > And as far as protecting democracy, I assure you that the geopolitical enemies of your nation either have similar models, or will have them very soon. There is very significant investment on ML models for text manipulation going on behind closed doors, funded by States. True, but it's a good idea to keep a high bar for developing and using it. There's a large difference between the resources of a nation state and various criminal enterprises. Per [0], GPT-3 took $12 million to train. That does not include the people with the relevant skills need to train it, and access to the compute hardware. [0]: https://venturebeat.com/2020/06/01/ai-machine-learning-openai-gpt-3-size-isnt-everything/ https://venturebeat.com/2020/06/01/ai-machine-learning-opena...
      - sudosysgen 6y ago
        I mean, I don't really see the benefit of limiting such a model to such famously reliable and disinterested actors as nation-states and multinational corporations. The kind of criminals that are going to threaten "democracy/social order/etc..." aren't going to be stopped by a price tag of 12 million dollars, or 120 million dollars for that matter. Just a single Mexican cartel would have the resources to pay for that, to say nothing of criminal groups that work in lockstep with states. The proceeds of crime are 900 billion dollars per year. Training GPT-3 is well within the means of dozens of criminal enterprises. But sure, we can praise OpenAI for protecting us from small enterprises and individuals, known to democracy, and instead reserving it for the use of megacorps and states, that are known not to threaten democracy or social order.
jchw 6y ago
(2019)
occamrazor 6y ago
OP refers to GPT-2. OpenAI has since then published both the code and the trained model.
- moonchild 6y ago
  From the gpt2 code release[1]: > We are still considering release of the larger models. Is there something I'm missing? 1. https://github.com/openai/gpt-2 https://github.com/openai/gpt-2
  - patresh 6y ago
    That part of the Readme seems to be out of date, they released the largest GPT-2 model last year https://www.openai.com/blog/gpt-2-1-5b-release/ https://www.openai.com/blog/gpt-2-1-5b-release/
    - gardenfelder 6y ago
      full reference: https://opencog.org/2020/07/virtual-opencogcon-july-15-16/ https://opencog.org/2020/07/virtual-opencogcon-july-15-16/
- nullc 6y ago
  They did, but now there is GPT3 which is closed access API only and I'm not aware of any stated plans to publish that model. On the plus side, if their claims about how GPT3 was constructed are truthful it should be possible for outside parties to reproduce it-- though at considerable cost.
- rvz 6y ago
  Well, this message didn't age well as soon as GPT-3 was made as an API access model only. It's getting harder to define 'OpenAI'. It is just another DeepMind, where they release the paper but never the model.
- garmaine 6y ago
  Look at the time stamps.
jcims 6y ago
One thing that frustrated me about this conversation at the time was that OpenAI explicitly stated right in the initial release announcement that that part of their rationale for the staged release process for GPT-2 was to force the conversation around releasing a powerful capability to the world. Basically a fire drill for the real thing. This was rarely acknowledged or discussed, it was almost entirely a combination of dunking on OpenAI for overhyping the product and excoriating them for interfering with open research. I don’t know that GPT-3 is at the level of material damage but it’s clearly moving in that direction. Watching someone interact with it that has a good sense for composing the right kind of prompt is a spine tingling experience. Given how poorly the PULSE demo that hallucinated fuzzy black folks into tan white people was received, it also makes PR sense not to release pretrained models that are obviously going to be easily tricked into saying terrible things. Exposing that as an API gives them the ability to police it a bit and study how the model behaves in the real world. Or it could just be a giant sellout and cash grab.
- unityByFreedom 6y ago
  > OpenAI explicitly stated right in the initial release announcement that that part of their rationale for the staged release process for GPT-2 was to force the conversation around releasing a powerful capability to the world. At founding, OpenAI was a non-profit and it said it would share any innovations openly. Now it is a for-profit and only releases teaser data/code.
- londons_explore 6y ago
  To be fair, with a multi-terabyte model, the number of people who have the money to make use of the model is probably only a few hundred...
  - dx034 6y ago
    Rather a few thousands at least. Most larger companies have that amount of data (if they can use the model is another question).
    - 6y ago
nexuist 6y ago
The problem is that OpenAI's motto is: "Discovering and enacting the path to safe artificial general intelligence." This does not mix well with very common AI tasks, such as facial recognition, deepfakes and deepnudes. With GPT-3 we are seeing levels of text comprehension and response that we've never seen in ML before. How can we abuse this? Well, we can conduct text-based gaslighting and manipulation on an unprecedented scale. Imagine choosing a Twitter profile to target, having the AI read every single tweet that person has made, and then DMing them insults or slurs that would most effectively apply to their personality. Imagine those Nigerian prince emails but each with a scam personally tailored to its audience, with any follow up questions answered with auto-generated but believable lies. Imagine GPT-3 being used to message young children on Instagram by the millions and tricking them into giving up personal information or images. I think, ultimately, there will be no such thing as "safe" AGI, because the intelligence we already have as humans can be and has been used to hurt others. OpenAI faces an existential crisis in this regards, and their best answer so far is simply to control access to the model so they can revoke it when bad actors are caught. This is something they can't do if the model is free and open source as we all want it to be.
- simiones 6y ago
  > With GPT-3 we are seeing levels of text comprehension and response that we've never seen in ML before. Is GPT-3 doing text comprehension, or just generation? I mean, can it extract facts/automatically tag text etc? Or "just" (I know how hard that is!) generate text with some similar characteristics/topics?
  - lanternfish 6y ago
    Yes. If you few-shot prime it with prompts of the form "<paragraph> tag:<tags>" and then just feed it "<novel paragraph> tag:" it'll generate <tags> automatically, just from completion. It does this shockingly well, the paper has specific examples. It can also do fun tricks like condensing a complicated Wikipedia article down to a second grade level, including built in metaphors. An example of this is on the API beta website IIRC.
gas9S9zw3P9c 6y ago
Someone asked for a more nuanced perspective, so here we go. For a lot of AI researchers, OpenAI has been a huge disappointment. We had hope that OpenAI would be the company to democratize AI with good open source work, transparency, no PR bullshit (aka DeepMind), and evangelism. That they would develop in the open, and perhaps even do research in the open. You know, kind of like the name says. It all started out okay with their release of OpenAI Gym, tutorials, leaderboards, and competitions around that. That was when Karpathy was still there. Over time, many projects have become abandoned, poorly maintained, or just disappeared [1]. And many projects they promised never happened [2]. OpenAI became just another research lab obsessed with publishing papers in closed (!) journals, indistinguishable from Google AI, DeepMind, FAIR, MSR, and the many others. There is nothing open or different about them. Most paper code is not published, and even when it is, it's just the typical poorly written and unmaintained research code that you see from other labs. None of their infrastructure is open source either, because it's needed to maintain their competitive advantage to train models and publish research papers. GPT-3 being offered as a paid API to a select number of people is latest joke in a long series of other jokes. All of this would be fine, if it was not for the name and branding of being a transparent and good-willed nonprofit company. It is just misleading and that rubs many people the wrong way, as if the whole "open" thing was just a PR stunt. HuggingFace [0] these days is pretty much what OpenAI should have been, but only time will tell what happens. [0] https://huggingface.co/ https://huggingface.co/ [1] https://www.reddit.com/r/MachineLearning/comments/aqwcyx/discussion_openai_should_now_change_their_name_to/egl03jv/ https://www.reddit.com/r/MachineLearning/comments/aqwcyx/dis... [2] https://github.com/openai/roboschool/issues/159 https://github.com/openai/roboschool/issues/159
- hsikka 6y ago
  A bunch of fellow researchers and I started Manifold Computing (https://manifoldcomputing.com https://manifoldcomputing.com), were we’re hoping to do live, open source research and build open source tools. As you said, time will tell but I hope we can do good work this way.
- screye 6y ago
  Hugging face and the Spacy team have done some incredible work. Huge fan of both for my NLP projects.
deleted 6y ago
[deleted]
vladimirsvsv77 6y ago
Agree
csomar 6y ago
I believe, having worked a little bit with GPT-2, that OpenAI is intentionally sabotaging access to their AI algorithm and data. They have started this sabotage with GPT-2 and with GPT-3 they simply didn't open source it. For GPT-2, their repository (https://github.com/openai/gpt-2 https://github.com/openai/gpt-2) is archived. Which is as good as saying that the project is abandoned and will receive no updates. The project already doesn't compile/run properly. This issue (https://github.com/openai/gpt-2/issues/178 https://github.com/openai/gpt-2/issues/178) could be solved relatively easily by either a one-line code fix, or merging a pull request (https://github.com/openai/gpt-2/pull/244/files https://github.com/openai/gpt-2/pull/244/files). This is not happening and I have a hard time to believe it is on good intentions. Oh, and by the way, I believe saying "This AI stuff is dangerous to world", the same as politicians saying "We need to check your web history for pedophilia stuff". It's funny how some people don't see the irony in how they oppose one thing and support another that is practically the same.
- Donthatme 6y ago
  I am not remotely in this field, and have not been following this closely at all. With that being said, what obligation do they have in maintaining GPT-2? Did they have some stated commitment that they walked back, or am I missing something else?
  - Frost1x 6y ago
    Their charter is here: https://openai.com/charter/ https://openai.com/charter/ >"We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research." So, in a perfect world you would not only publish your research but also the code that is fundamental to it. Maintaining or abandoning research code on the other hand is an entirely different (costly) story that's simply an artifact of research focused software development. It is typically abandoned. Personally, I see a huge flaw in the underlying philosophy. This presumption that this specific organization is or can somehow be benevolent flies in the face of all history. With nuclear weapons, most of the scientists regretted supporting their countries regardless of how beneveloant they thought they were. In general, any sort of concentrated power tends to corrupt. It takes a very special mindset to understand power and refuse to abuse it. I'm not sure this is something that can be easily learned, trained, or you could expect a large group with access to the power to all adhere to the principles of.
vffhfhf 6y ago
They think they are so good. Their hype generator if released would collapse our whole society. How high are they? Or are they just dense?
longtom 6y ago
I have long suspected that they are already being tightly controlled by the U.S. military. They obviously know what is at stake and they logically need to maintain AI supremacy. We are amidst of an AI arms race and it won't stop until we reach singularity or people start nuking each other to thwart a major cleansing operation with slaughter nanobots.
ypcx 6y ago
GPT-3 is (the process of) general AI being born in front of our eyes, catching us unprepared, making us realize that the we may be a bit rigid in understanding of what intelligence really means. If primed correctly, can GPT-3 produce worthwhile improvements to its own paper or implementation? Can GPT-3 be made to prime and query itself in a loop, by asking it to produce a better query for itself? Can its knowledge be already used to disrupt existing frontiers of human thought? The simple and safe answer here is that we don't know yet. In a world full of ideological adversity and cold-blooded corporate competition, slowing down the release of technology which may or may not be capable of a runaway effect (a.k.a. technological singularity) in the hands of an adversary is an intelligent and prudent thing to do.
- metafunctor 6y ago
  Nope. The simple and safe answer to all those questions is certainly “no”. GPT-3 not intelligent in the way we most of us would describe intelligence.
  - icebergwarrior 6y ago
    I don't think you can make such definitive statements. GPT-3 is certainly intelligent the way a lot of us would describe intelligence. It can produce content in a way this is indistinguishable from humans. We don't know what else it can do. We don't know the pace of improvements happening here. There are a lot of open questions.
Nuzzerino 6y ago
There's always OpenCog: https://github.com/opencog https://github.com/opencog
- mindcrime 6y ago
  Now is an especially good time to join the OpenCog project. Ben and the folks behind it are just now starting a big push to re-architect big chunks of it, including the AtomSpace. And OpenCogCon was just held as a two day virtual conference, and the entire thing is available on Youtube. I'd encourage anyone who's interested to join the Opencog Slack and/or Google group and check out what's going on.
  - gardenfelder 6y ago
    Day 1: https://www.youtube.com/watch?v=iZOhZFd52y4 https://www.youtube.com/watch?v=iZOhZFd52y4
Kutta 6y ago
OpenAI should have never been open, named "OpenAI", or advertised itself as being open. At the time of OpenAI's inception, much of the AI risk community deemed it as harmful, although that wasn't spoken out a lot, because it is a delicate affair to criticize misguided effort on AI safety when the status quo was almost no effort being spent on AI safety at all. It is a minor consolation that OpenAI turned out to be less open than initially advertised. https://web.archive.org/web/20200518081726/https://slatestarcodex.com/2015/12/17/should-ai-be-open/ https://web.archive.org/web/20200518081726/https://slatestar...
- AlexandrB 6y ago
  So "open, like a gate" not "open, like a public resource".
  - non-entity 6y ago
    Somewhat random but this debate about the "Open" in OpenAI's name reminds me of OpenVMS which is not open source, but supports "open systems" like POSIX.
sharker8 6y ago
How are people getting gpt-3 api keys? Might I have one?
minhazm 6y ago
One of the early stated goals of OpenAI was to advance AI but do so safely. Can you imagine if scammers and "bad actors" got access to GPT-3? You could create such sophisticated and targeted scams. Things could get extremely ugly fast. The choice to limit access to this seems to be completely in line with their original values.
r34 6y ago
I'm not surprised. Since I've seen S. Altman talking in one of his videos about "things that you (the employer) can't tell to your employees" I completely don't trust that guy.
- tuanvc 6y ago
  Any video link?
dandanua 6y ago
It's inevitable that AI research will be more and more closed. AI is a weapon which is potentially much more dangerous than nuclear. It's naive to think that the most dangerous weapon will be accessible to a general public.
- vasili111 6y ago
  AI is dangerous when it is closed and inaccesseble to majority of people. Lets take deepfake. If it was closed and inaccessible to majority of people it could be used as a powerful weapon (making influence on politics, etc). Now when everyone has access to it and aware that that kind of fakes is not hard to make, it is not a weapon anymore. Of course you can still use it as a weapon, especially in not well developed countries, but even there after some time it will become useless as a weapon too. I think person who made deepfake saved many people lives and that is the best approach to deal with so called "danger of AI".
  - dandanua 6y ago
    I don't see how deepfake is less dangerous through it availability. Because everyone is now aware? That doesn't mean this tool has to be open. Also no one is making a proper fact-check nowadays and a deepfake attack can do a lot of harm anyway. Or maybe because you can retaliate with counter-deepfake? Well, that's just ridiculous. What I see is that deepfake is used for a fake-porn videos with a known actresses or some ordinary people for harassing and blackmailing them. I don't see how its availability saves lives.
    - vasili111 6y ago
      Because people are now aware that that kind of fakes exist and it is much harder to use fakes in more serious things like politics.
throwaway7281 6y ago
One good thing about this is: An actually free and open ecosystem will emerge - it will take time and countless people-hours but it will happen and it will prevail.
deleted 6y ago
[deleted]
eaxitect 6y ago
OpenAI is open as businesses are open.
thedudeabides5 6y ago
Funny how there's so much emotional processing here from the open-source community. What started as a techno-utopian non-profit project has transformed into a well-branded very much for profit corporation, right under their noses! I say good for them. Look forward to seeing how the tech plays out.

503 points by gitgud 6y ago

Can anyone point to a more nuanced perspective on this? Ideally not only with regards to OpenAI; it is still quite common that research published in some of the "top" venues does not disclose the underlying source code. Anecdote: I once pushed researchers to publish their code when I was peer-reviewing and it turned out that the code was super slow in comparison to the benchmark algorithms they compared with (comparison only included accuracy etc, not speed), something that I am sure the authors were aware of, but chose to not report in the paper.

unityByFreedom 6y ago
> it is still quite common that research published in some of the "top" venues does not disclose the underlying source code. The whole point of "OpenAI" was to share their innovations openly. They were a non profit. Now they're a "capped profit" and not sharing what's needed to reproduce their results (full code and training data). They acquired talent on a bunch of good will and shifted into a for profit.
IAmEveryone 6y ago
I don't necessarily agree with the following with regards to OpenAI, but it is what I have been told by a few professors and other academics in bioinformatics: Money is terribly tight in academia, and even at hot think tanks like OpenAI. The difference between a working but not very pretty prototype and a polished product you are willing to be associated with may seem like just a bunch of trivial boilerplate such as documentation or tests. It is indeed boring work (at least for scientists), but a lot of it. And if the intention is to publish and support something on an ongoing basis, and for widespread use, you probably need to invest 5x to 10x as much time and money. So in the end most code in academia is hidden because it's hideous, not to hide how they hardcoded all the good jokes GTP-3 comes up with.
- woah 6y ago
  Such an odd attitude. There’s no way it takes 5-10x engineering effort to write some tests for components of a one-shot python script, as most scientific code is. It’s maybe the easiest type of software to test. Would this type of attitude fly for anything else? Can you imagine if chemists worked like this? “Yea most chemists don’t like to release their procedures, since taking the trouble of accurately measuring out reagents could be 5-10x the effort and grant money is tight. Plus sometimes they don’t want to go all the way to the cabinet to grab a beaker so they just do the reaction in a coffee mug. The truth is that sometimes the procedure is just too ugly to release. And if they come out of the lab with a new chemical, what does it matter?”
  - mennis16 6y ago
    I mean a lot of biology papers are loose on protocol details to be honest, dunno about Chemistry but for Bio it can often be difficult to exactly replicate a method because of lack of details alone. That said I think this should change given how easy it is these days to share supplemental info online. As far as ugly code I wouldn't be surprised if a lot of academics won't take the time to fix it up, but that doesn't mean it shouldn't be released IMO. Maybe an open source community would pop up that could help clean this code for them.
albntomat0 6y ago
Here's my alternative view: If OpenAI had open sourced GPT-3, there would be an equivalently angry thread about how they were endangering democracy/social order/etc, and not being responsible with the powerful tool they had created (see other threads on HN regarding those working in facial recognition). Both that group, and the one posting here have valid points, and there would be strong, valid critique of their decision no matter which way they chose.
- sudosysgen 6y ago
  But we do realize that sooner or later a model even more powerful will be released to the public, right? And as far as protecting democracy, I assure you that the geopolitical enemies of your nation either have similar models, or will have them very soon. There is very significant investment on ML models for text manipulation going on behind closed doors, funded by States.
  - albntomat0 6y ago
    > But we do realize that sooner or later a model even more powerful will be released to the public, right? And at that time, we can equally criticize whomever releases it. > And as far as protecting democracy, I assure you that the geopolitical enemies of your nation either have similar models, or will have them very soon. There is very significant investment on ML models for text manipulation going on behind closed doors, funded by States. True, but it's a good idea to keep a high bar for developing and using it. There's a large difference between the resources of a nation state and various criminal enterprises. Per [0], GPT-3 took $12 million to train. That does not include the people with the relevant skills need to train it, and access to the compute hardware. [0]: https://venturebeat.com/2020/06/01/ai-machine-learning-openai-gpt-3-size-isnt-everything/ https://venturebeat.com/2020/06/01/ai-machine-learning-opena...
    - sudosysgen 6y ago
      I mean, I don't really see the benefit of limiting such a model to such famously reliable and disinterested actors as nation-states and multinational corporations. The kind of criminals that are going to threaten "democracy/social order/etc..." aren't going to be stopped by a price tag of 12 million dollars, or 120 million dollars for that matter. Just a single Mexican cartel would have the resources to pay for that, to say nothing of criminal groups that work in lockstep with states. The proceeds of crime are 900 billion dollars per year. Training GPT-3 is well within the means of dozens of criminal enterprises. But sure, we can praise OpenAI for protecting us from small enterprises and individuals, known to democracy, and instead reserving it for the use of megacorps and states, that are known not to threaten democracy or social order.

jchw 6y ago

(2019)

occamrazor 6y ago

OP refers to GPT-2. OpenAI has since then published both the code and the trained model.

moonchild 6y ago
From the gpt2 code release[1]: > We are still considering release of the larger models. Is there something I'm missing? 1. https://github.com/openai/gpt-2 https://github.com/openai/gpt-2
- patresh 6y ago
  That part of the Readme seems to be out of date, they released the largest GPT-2 model last year https://www.openai.com/blog/gpt-2-1-5b-release/ https://www.openai.com/blog/gpt-2-1-5b-release/
  - gardenfelder 6y ago
    full reference: https://opencog.org/2020/07/virtual-opencogcon-july-15-16/ https://opencog.org/2020/07/virtual-opencogcon-july-15-16/
nullc 6y ago
They did, but now there is GPT3 which is closed access API only and I'm not aware of any stated plans to publish that model. On the plus side, if their claims about how GPT3 was constructed are truthful it should be possible for outside parties to reproduce it-- though at considerable cost.
rvz 6y ago
Well, this message didn't age well as soon as GPT-3 was made as an API access model only. It's getting harder to define 'OpenAI'. It is just another DeepMind, where they release the paper but never the model.
garmaine 6y ago
Look at the time stamps.

jcims 6y ago

One thing that frustrated me about this conversation at the time was that OpenAI explicitly stated right in the initial release announcement that that part of their rationale for the staged release process for GPT-2 was to force the conversation around releasing a powerful capability to the world. Basically a fire drill for the real thing. This was rarely acknowledged or discussed, it was almost entirely a combination of dunking on OpenAI for overhyping the product and excoriating them for interfering with open research. I don’t know that GPT-3 is at the level of material damage but it’s clearly moving in that direction. Watching someone interact with it that has a good sense for composing the right kind of prompt is a spine tingling experience. Given how poorly the PULSE demo that hallucinated fuzzy black folks into tan white people was received, it also makes PR sense not to release pretrained models that are obviously going to be easily tricked into saying terrible things. Exposing that as an API gives them the ability to police it a bit and study how the model behaves in the real world. Or it could just be a giant sellout and cash grab.

unityByFreedom 6y ago
> OpenAI explicitly stated right in the initial release announcement that that part of their rationale for the staged release process for GPT-2 was to force the conversation around releasing a powerful capability to the world. At founding, OpenAI was a non-profit and it said it would share any innovations openly. Now it is a for-profit and only releases teaser data/code.
londons_explore 6y ago
To be fair, with a multi-terabyte model, the number of people who have the money to make use of the model is probably only a few hundred...
- dx034 6y ago
  Rather a few thousands at least. Most larger companies have that amount of data (if they can use the model is another question).
  - 6y ago

nexuist 6y ago

The problem is that OpenAI's motto is: "Discovering and enacting the path to safe artificial general intelligence." This does not mix well with very common AI tasks, such as facial recognition, deepfakes and deepnudes. With GPT-3 we are seeing levels of text comprehension and response that we've never seen in ML before. How can we abuse this? Well, we can conduct text-based gaslighting and manipulation on an unprecedented scale. Imagine choosing a Twitter profile to target, having the AI read every single tweet that person has made, and then DMing them insults or slurs that would most effectively apply to their personality. Imagine those Nigerian prince emails but each with a scam personally tailored to its audience, with any follow up questions answered with auto-generated but believable lies. Imagine GPT-3 being used to message young children on Instagram by the millions and tricking them into giving up personal information or images. I think, ultimately, there will be no such thing as "safe" AGI, because the intelligence we already have as humans can be and has been used to hurt others. OpenAI faces an existential crisis in this regards, and their best answer so far is simply to control access to the model so they can revoke it when bad actors are caught. This is something they can't do if the model is free and open source as we all want it to be.

simiones 6y ago
> With GPT-3 we are seeing levels of text comprehension and response that we've never seen in ML before. Is GPT-3 doing text comprehension, or just generation? I mean, can it extract facts/automatically tag text etc? Or "just" (I know how hard that is!) generate text with some similar characteristics/topics?
- lanternfish 6y ago
  Yes. If you few-shot prime it with prompts of the form "<paragraph> tag:<tags>" and then just feed it "<novel paragraph> tag:" it'll generate <tags> automatically, just from completion. It does this shockingly well, the paper has specific examples. It can also do fun tricks like condensing a complicated Wikipedia article down to a second grade level, including built in metaphors. An example of this is on the API beta website IIRC.

gas9S9zw3P9c 6y ago

Someone asked for a more nuanced perspective, so here we go. For a lot of AI researchers, OpenAI has been a huge disappointment. We had hope that OpenAI would be the company to democratize AI with good open source work, transparency, no PR bullshit (aka DeepMind), and evangelism. That they would develop in the open, and perhaps even do research in the open. You know, kind of like the name says. It all started out okay with their release of OpenAI Gym, tutorials, leaderboards, and competitions around that. That was when Karpathy was still there. Over time, many projects have become abandoned, poorly maintained, or just disappeared [1]. And many projects they promised never happened [2]. OpenAI became just another research lab obsessed with publishing papers in closed (!) journals, indistinguishable from Google AI, DeepMind, FAIR, MSR, and the many others. There is nothing open or different about them. Most paper code is not published, and even when it is, it's just the typical poorly written and unmaintained research code that you see from other labs. None of their infrastructure is open source either, because it's needed to maintain their competitive advantage to train models and publish research papers. GPT-3 being offered as a paid API to a select number of people is latest joke in a long series of other jokes. All of this would be fine, if it was not for the name and branding of being a transparent and good-willed nonprofit company. It is just misleading and that rubs many people the wrong way, as if the whole "open" thing was just a PR stunt. HuggingFace [0] these days is pretty much what OpenAI should have been, but only time will tell what happens. [0] https://huggingface.co/ https://huggingface.co/ [1] https://www.reddit.com/r/MachineLearning/comments/aqwcyx/discussion_openai_should_now_change_their_name_to/egl03jv/ https://www.reddit.com/r/MachineLearning/comments/aqwcyx/dis... [2] https://github.com/openai/roboschool/issues/159 https://github.com/openai/roboschool/issues/159

hsikka 6y ago
A bunch of fellow researchers and I started Manifold Computing (https://manifoldcomputing.com https://manifoldcomputing.com), were we’re hoping to do live, open source research and build open source tools. As you said, time will tell but I hope we can do good work this way.
screye 6y ago
Hugging face and the Spacy team have done some incredible work. Huge fan of both for my NLP projects.

deleted 6y ago

[deleted]

vladimirsvsv77 6y ago

Agree

csomar 6y ago

I believe, having worked a little bit with GPT-2, that OpenAI is intentionally sabotaging access to their AI algorithm and data. They have started this sabotage with GPT-2 and with GPT-3 they simply didn't open source it. For GPT-2, their repository (https://github.com/openai/gpt-2 https://github.com/openai/gpt-2) is archived. Which is as good as saying that the project is abandoned and will receive no updates. The project already doesn't compile/run properly. This issue (https://github.com/openai/gpt-2/issues/178 https://github.com/openai/gpt-2/issues/178) could be solved relatively easily by either a one-line code fix, or merging a pull request (https://github.com/openai/gpt-2/pull/244/files https://github.com/openai/gpt-2/pull/244/files). This is not happening and I have a hard time to believe it is on good intentions. Oh, and by the way, I believe saying "This AI stuff is dangerous to world", the same as politicians saying "We need to check your web history for pedophilia stuff". It's funny how some people don't see the irony in how they oppose one thing and support another that is practically the same.

Donthatme 6y ago
I am not remotely in this field, and have not been following this closely at all. With that being said, what obligation do they have in maintaining GPT-2? Did they have some stated commitment that they walked back, or am I missing something else?
- Frost1x 6y ago
  Their charter is here: https://openai.com/charter/ https://openai.com/charter/ >"We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research." So, in a perfect world you would not only publish your research but also the code that is fundamental to it. Maintaining or abandoning research code on the other hand is an entirely different (costly) story that's simply an artifact of research focused software development. It is typically abandoned. Personally, I see a huge flaw in the underlying philosophy. This presumption that this specific organization is or can somehow be benevolent flies in the face of all history. With nuclear weapons, most of the scientists regretted supporting their countries regardless of how beneveloant they thought they were. In general, any sort of concentrated power tends to corrupt. It takes a very special mindset to understand power and refuse to abuse it. I'm not sure this is something that can be easily learned, trained, or you could expect a large group with access to the power to all adhere to the principles of.

vffhfhf 6y ago

They think they are so good. Their hype generator if released would collapse our whole society. How high are they? Or are they just dense?

longtom 6y ago

I have long suspected that they are already being tightly controlled by the U.S. military. They obviously know what is at stake and they logically need to maintain AI supremacy. We are amidst of an AI arms race and it won't stop until we reach singularity or people start nuking each other to thwart a major cleansing operation with slaughter nanobots.

ypcx 6y ago

GPT-3 is (the process of) general AI being born in front of our eyes, catching us unprepared, making us realize that the we may be a bit rigid in understanding of what intelligence really means. If primed correctly, can GPT-3 produce worthwhile improvements to its own paper or implementation? Can GPT-3 be made to prime and query itself in a loop, by asking it to produce a better query for itself? Can its knowledge be already used to disrupt existing frontiers of human thought? The simple and safe answer here is that we don't know yet. In a world full of ideological adversity and cold-blooded corporate competition, slowing down the release of technology which may or may not be capable of a runaway effect (a.k.a. technological singularity) in the hands of an adversary is an intelligent and prudent thing to do.

metafunctor 6y ago
Nope. The simple and safe answer to all those questions is certainly “no”. GPT-3 not intelligent in the way we most of us would describe intelligence.
- icebergwarrior 6y ago
  I don't think you can make such definitive statements. GPT-3 is certainly intelligent the way a lot of us would describe intelligence. It can produce content in a way this is indistinguishable from humans. We don't know what else it can do. We don't know the pace of improvements happening here. There are a lot of open questions.

Nuzzerino 6y ago

There's always OpenCog: https://github.com/opencog https://github.com/opencog

mindcrime 6y ago
Now is an especially good time to join the OpenCog project. Ben and the folks behind it are just now starting a big push to re-architect big chunks of it, including the AtomSpace. And OpenCogCon was just held as a two day virtual conference, and the entire thing is available on Youtube. I'd encourage anyone who's interested to join the Opencog Slack and/or Google group and check out what's going on.
- gardenfelder 6y ago
  Day 1: https://www.youtube.com/watch?v=iZOhZFd52y4 https://www.youtube.com/watch?v=iZOhZFd52y4

Kutta 6y ago

OpenAI should have never been open, named "OpenAI", or advertised itself as being open. At the time of OpenAI's inception, much of the AI risk community deemed it as harmful, although that wasn't spoken out a lot, because it is a delicate affair to criticize misguided effort on AI safety when the status quo was almost no effort being spent on AI safety at all. It is a minor consolation that OpenAI turned out to be less open than initially advertised. https://web.archive.org/web/20200518081726/https://slatestarcodex.com/2015/12/17/should-ai-be-open/ https://web.archive.org/web/20200518081726/https://slatestar...

AlexandrB 6y ago
So "open, like a gate" not "open, like a public resource".
- non-entity 6y ago
  Somewhat random but this debate about the "Open" in OpenAI's name reminds me of OpenVMS which is not open source, but supports "open systems" like POSIX.

sharker8 6y ago

How are people getting gpt-3 api keys? Might I have one?

minhazm 6y ago

One of the early stated goals of OpenAI was to advance AI but do so safely. Can you imagine if scammers and "bad actors" got access to GPT-3? You could create such sophisticated and targeted scams. Things could get extremely ugly fast. The choice to limit access to this seems to be completely in line with their original values.

r34 6y ago

I'm not surprised. Since I've seen S. Altman talking in one of his videos about "things that you (the employer) can't tell to your employees" I completely don't trust that guy.

tuanvc 6y ago
Any video link?

dandanua 6y ago

It's inevitable that AI research will be more and more closed. AI is a weapon which is potentially much more dangerous than nuclear. It's naive to think that the most dangerous weapon will be accessible to a general public.

vasili111 6y ago
AI is dangerous when it is closed and inaccesseble to majority of people. Lets take deepfake. If it was closed and inaccessible to majority of people it could be used as a powerful weapon (making influence on politics, etc). Now when everyone has access to it and aware that that kind of fakes is not hard to make, it is not a weapon anymore. Of course you can still use it as a weapon, especially in not well developed countries, but even there after some time it will become useless as a weapon too. I think person who made deepfake saved many people lives and that is the best approach to deal with so called "danger of AI".
- dandanua 6y ago
  I don't see how deepfake is less dangerous through it availability. Because everyone is now aware? That doesn't mean this tool has to be open. Also no one is making a proper fact-check nowadays and a deepfake attack can do a lot of harm anyway. Or maybe because you can retaliate with counter-deepfake? Well, that's just ridiculous. What I see is that deepfake is used for a fake-porn videos with a known actresses or some ordinary people for harassing and blackmailing them. I don't see how its availability saves lives.
  - vasili111 6y ago
    Because people are now aware that that kind of fakes exist and it is much harder to use fakes in more serious things like politics.

throwaway7281 6y ago

One good thing about this is: An actually free and open ecosystem will emerge - it will take time and countless people-hours but it will happen and it will prevail.

deleted 6y ago

[deleted]

eaxitect 6y ago

OpenAI is open as businesses are open.

thedudeabides5 6y ago

Funny how there's so much emotional processing here from the open-source community. What started as a techno-utopian non-profit project has transformed into a well-branded very much for profit corporation, right under their noses! I say good for them. Look forward to seeing how the tech plays out.