3 ms·
Some people are noting that TensorFlow evens the playing field, allowing other companies to poach Google's employees. Whilst that is true to some light degree,
by Smerity 10y ago
Some people are noting that TensorFlow evens the playing field, allowing other companies to poach Google's employees. Whilst that is true to some light degree, there are amazingly large caveats - Google still have the major advantage when it comes to poaching. The other important note is that open sourcing TensorFlow is likely more important for attracting recent graduates in machine learning than it is for attracting employees of other companies.
Most machine learning engineers and researchers get excited by two things: underlying hardware and underlying tooling.
Google have enough GPUs to throw 50 K80s for a few weeks into the hands of a single researcher for a single paper[1]. That's impressive. It gets even more impressive considering Google are making custom made ASICs that are more efficient - both power and computation - than GPUs too. None of that hardware is open source and even if it was, I don't see anyone getting them into general production in the near future. Nervana[2] are the closest to custom hardware I know about yet still have many steps until production.
TensorFlow is also just the beginnings of the tooling Google has internally. The original TensorFlow whitepaper talks about their internal distributed training stack. A tiny recreation of this has reached the wild but it misses many of the interesting optimizations. (1) Automated efficient allocation of tensor operations to components (does this op go on GPU? CPU? Across the network to another GPU machine?). (2) Minor optimizations, such as truncating 32 bit floats to 16, then re-expanding - they noted it in the paper but I've not seen external verification. (3) Google DeepMind moved to TensorFlow but given they do a lot of reinforcement learning, they likely have a modified TF internally that's better tailored for RL. (4) etc etc etc
Given the majority of the contributors to TensorFlow are also from Google, that gives them the strongest say in the future of the framework. This is especially important given that Google would likely be trying to ensure their internal modifications and variations of TensorFlow remain in line with the open version. For that reason, I don't expect to see TensorFlow handed off to a third party caretaker any time soon.
Releasing TensorFlow is a net win for Google, both in "making the world a better place" and "making Google a better place (for attracting talent)".
[1]: https://arxiv.org/abs/1607.05691 https://arxiv.org/abs/1607.05691
[2]: https://www.nervanasys.com/ https://www.nervanasys.com/