5 ms·
It's somewhat disappointing that research is the primary motivator for the switch. PyTorch still has a ways to go in tooling for toy usage of models and deploym
by minimaxir 7y ago
It's somewhat disappointing that research is the primary motivator for the switch. PyTorch still has a ways to go in tooling for toy usage of models and deployment of models to production compared to TensorFlow (incidentally, GPT-2, the most public of OpenAI's released models, uses TensorFlow 1.X as a base). For AI newbies, I've seen people recommend PyTorch over TensorFlow just because "all the big players are using it," without listing the caveats.
The future of AI research will likely be interoperability between multiple frameworks to support both needs (e.g. HuggingFace Transformers which started as PyTorch-only but now also supports TF 2.X with relative feature parity).
- Nimitz14 7y agoOpenAI doesn't have any products right? So research is a lot more important than production for them. Surprises me it took them this long.
- minimaxir 7y agoThe aforementioned GPT-2 and Spinning Up were released, open-sourced tools using TF; however, other recent models such as the OpenAI Five Dota 2 bot, the robot dexterity demo, and the hide-and-seek simulation were not open-sourced. If OpenAI is signaling that they are changing their open-source strategy, they should be more explicit about that.
- Voloskaya 7y ago> It's somewhat disappointing that research is the primary motivator for the switch. They are a research organization, how is that disappointing?
- minimaxir 7y agoMaking AI more open is synonymous with making AI more accessible, which (IMO) is much better facilitated with TensorFlow/Keras versus PyTorch. Many AI tutorials imply that the more complicated an AI approach is, the more effective it is, which isn't practical, especially for newbies without a deep background.
- eachro 7y agoAccessible to whom? What makes tensorflow/keras more accessible than pytorch?
- minimaxir 7y agoAccessible to non-researchers, especially those with a programming background but not an AI background. The TF/Keras approach advocates the minimum amount of code necessary and effort needed to make model changes, with sensible default configurations and layer architectures.
- octbash 7y agoMinimum=/=lowest effort. Especially with the caveat of "with a programming background", it is far easier to reason and debug through PyTorch with just Python knowledge, compared to TensorFlow/Keras, which sooner or later requires you to learn a condensed history of TensorFlow/Keras development to understand why things are the way they are. In my opinion, import lib lib.train("imagenet", "resnet50", epochs=10) lib.eval() is NOT a good example of a beginner friendly library. It's a thin wrapper facade that hides all of the actual complexity behind "Train ImageNet in 3 lines of code!"
- minimaxir 7y agoFair; maybe minimum isn't the right word. More like "minimum without full abstraction." The Keras examples are a good reference (e.g. https://www.tensorflow.org/tutorials/keras/classification https://www.tensorflow.org/tutorials/keras/classification ); even without an AI background, you have a sense of both what's going on how to tweak the model to improve it.
- tastyminerals 7y agoThe reason why Keras became so popular is that it borrowed a lot of concepts from Lua Torch (which predates even Theano). And anyone who worked with Torch immediately sees it reading Keras code. But Torch was Lua and naturally it received less recognition than it deserved. Your will not lose anything by simply moving to PyTorch.
- bryanhpchiang 7y agoOpenAI is a fundamentally a research laboratory - their top priority is pushing the envelope on machine learning research and releasing their progress responsibly. Anything that reduces experimentation and research speed should be avoided, thus the switch to PyTorch makes sense. As long as as OpenAI is open sourcing their work, there will always be others that will port it over to other frameworks.