5 ms·
One thing that frustrated me about this conversation at the time was that OpenAI explicitly stated right in the initial release announcement that that part of t
by jcims 6y ago
One thing that frustrated me about this conversation at the time was that OpenAI explicitly stated right in the initial release announcement that that part of their rationale for the staged release process for GPT-2 was to force the conversation around releasing a powerful capability to the world. Basically a fire drill for the real thing. This was rarely acknowledged or discussed, it was almost entirely a combination of dunking on OpenAI for overhyping the product and excoriating them for interfering with open research.
I don’t know that GPT-3 is at the level of material damage but it’s clearly moving in that direction. Watching someone interact with it that has a good sense for composing the right kind of prompt is a spine tingling experience.
Given how poorly the PULSE demo that hallucinated fuzzy black folks into tan white people was received, it also makes PR sense not to release pretrained models that are obviously going to be easily tricked into saying terrible things. Exposing that as an API gives them the ability to police it a bit and study how the model behaves in the real world.
Or it could just be a giant sellout and cash grab.
- unityByFreedom 6y ago> OpenAI explicitly stated right in the initial release announcement that that part of their rationale for the staged release process for GPT-2 was to force the conversation around releasing a powerful capability to the world. At founding, OpenAI was a non-profit and it said it would share any innovations openly. Now it is a for-profit and only releases teaser data/code.
- londons_explore 6y agoTo be fair, with a multi-terabyte model, the number of people who have the money to make use of the model is probably only a few hundred...
- dx034 6y agoRather a few thousands at least. Most larger companies have that amount of data (if they can use the model is another question).
- londons_explore 6y agoYep, but running inference on it at any reasonable performance requires you to have all of it in GPU RAM - Ie. you need a cluster of ~100 high performance GPU's.
- trsohmers 6y agoThe largest version of GPT-3 is 175B parameters, which is ~350GB. I frequently use 8x and 10x RTX8000 boxes (and can access a 16x JBOG system as well), and the 8x system would have 384GB of VRAM. These sort of systems start off at only ~$60k.