3 ms·
Hi! OpenAI employee and paper co-author here, speaking entirely on my own (nobody else in the company knows I'm posting this). Occam's razor is a great princi
by wuthefwasthat 8y ago
Hi! OpenAI employee and paper co-author here, speaking entirely on my own (nobody else in the company knows I'm posting this). Occam's razor is a great principle, and I completely see how it might point towards your viewpoint. But I'd like to make a few points which will hopefully correct some misconceptions:
1. When we made the decision to do partial release, the talking points were about dangers and benefits of release - reputational benefits was one of the four main benefits we listed and not talked about much. Everyone in the room, including mostly technical non-comms folk, agreed that caution was a reasonable decision, given our uncertainty about malicious use.
2. I was at every meeting we had with the press, prior to release. We had written and rehearsed our major talking points beforehand (and stuck to them). Here they are: what we did technically, how it's different/new, how it's still limited; generality and zero-shot behavior of the model; how the trends of scaling are steady; the realism of the synthetic text, and how we're potentially on a trajectory similar to the synthetic images (we specifically compared GPT-2 to Alec's 2015 work on DCGAN, not the photorealistic stuff we see now); malicious use cases and policy implications; framing our partial release as an experiment and a discussion-starter. Ultimately, we didn't have fine-grained control over the actual articles written.
3. My impression is our samples are a qualitative jump from anything publicly known prior to our release. Most people (whether academics or journalists) who have seen our demo come away impressed; a small number had more muted responses. I would be pretty surprised if there weren't use cases this immediately enabled that previous language models didn't; whether they are more economically-valuable than malicious, I'm unsure (although I have a guess).
4. I personally don't have any evidence that the people I worked with on the release aren't well-intentioned, aren't thoughtful, or aren't smart.
Stepping back a bit, there are definitely aspects of what we did that I already regret, and perhaps more aspects I will regret in the future. I wish we'd had more time to think through every aspect of release, but opening up discussion to the community doesn't seem like a terrible outcome, regardless of how well thought out our decisions were.