5 ms·
I'll bet you $20 that GPT-3 will be in Pytorch. My point is that we probably won't be seeing more awesome projects from OpenAI written in Tensorflow. And that'
by sillysaurusx 7y ago
I'll bet you $20 that GPT-3 will be in Pytorch.
My point is that we probably won't be seeing more awesome projects from OpenAI written in Tensorflow. And that's unfortunate. Memory saving gradients were particularly helpful.
- chillee 7y agoI mean, this starts to sound like your complaint isn't that OpenAI is standardizing as a framework but that the framework is PyTorch :)
- SpaceManNabs 7y agoThat is usually the case with framework wars.
- octbash 7y agoMy counter-arguments (as a huge PyTorch fan) are: 1. GPT hasn't really been about model/architectural experimentation, just scale. GPT-2 and GPT were architecturally very similar. Scale, especially at the scale of GPT-*, is one avenue that TensorFlow does have an edge over PyTorch 2. Work on GPT-3 probably started quite a while ago.