3 ms·
This is a very interesting claim. I find it credible because it stands to reason that projects like DeepSpeed[1] and TorchScript[2] wouldn't need to exist if in
by dwrodri 6y ago
This is a very interesting claim. I find it credible because it stands to reason that projects like DeepSpeed[1] and TorchScript[2] wouldn't need to exist if inference performance of research PyTorch models was satisfactory for production, but often case it isn't.
It appears as though Flashlight is built on ArrayFire. I haven't seen how gradients are managed in arrayfire-ml, but perhaps it is the case that the autograd implementation in PyTorch was a bottleneck and this is a ground up approach.
Editing as I didn't address your second point. I can neither confirm or deny the motivations for creating Torch being related to FB's desire to depend on an Alphabet-managed codebase. I know there are lots of reasons why programmers prefer the UX of the PyTorch Python API (I do as well), and there are probably other reasons I can't recall off the top of my head. I am only saying that PyTorch is contributing to the ML ecosystem already by the sole virtue that it isn't a Google product.
[1] = https://github.com/microsoft/DeepSpeed https://github.com/microsoft/DeepSpeed
[2] = https://pytorch.org/docs/stable/jit.html https://pytorch.org/docs/stable/jit.html
- ipsum2 6y agoPyTorch is based off of Torch, which was first released in 2002, predating TensorFlow by 13 years. https://en.wikipedia.org/wiki/Torch_(machine_learning) https://en.wikipedia.org/wiki/Torch_(machine_learning)
- liuliu 6y agoTorch (Lua) predates TensorFlow and is Lecun's pet project for a few years at that point already. But Lua as a language is unpopular at a time. PyTorch would be a welcome addition then. But even if no PyTorch (nor Caffe2) in an alternative timeline, I would imagine FB would be stuck with Lua Torch for quite some time.
- tastyminerals2 6y agoDon’t forget that Lua Torch backend is C while Pytorch is C++ mostly.