4 ms·

A big lesson I learned from PyTorch vs other frameworks is that productivity trumps incremental performance improvement. Both Caffe and MXNet marketed themselve

A big lesson I learned from PyTorch vs other frameworks is that productivity trumps incremental performance improvement. Both Caffe and MXNet marketed themselves for being fast, yet apparently being faster here and here by some percentage simply didn't matter that much. On the other hand, once we make a system work and make it popular, the community will close the performance gap sooner than competitors expect. Another lesson is probably old but worth repeating: investment and professional polishing matters to open source projects. Meta reportedly had more than 300 (?) people working on PyTorch, helping the community resolve issues, producing tons of high-quality documentation and libraries, and marketing themselves nicely in all kinds of conferences and media.

t-vi 4y ago
> Another lesson is probably old but worth repeating: investment and professional polishing matters to open source projects While I'm not going to disagree that it matters, to my mind the main thing about early (0.x, 1.x) PyTorch's success was a clear vision (most visible to me from Soumith and Adam) that put the targeted users first (as in your productivity comment), awesome execution on it, and a creating a community where I, as an outside contributor with modes skill and no AI track record, and likely many others, felt welcome. Now, the many great people employed by Meta, but also other players like MS, NVidia, AMD, Intel, ... to work on PyTorch enabled all the things that make PyTorch 2.0 nicer and more broadly applicable than 1.0. (And to my mind it's quite a non-trivial accomplishment to enable other corporate players to join the party.)
SleekEagle 4y ago
Exactly, especially in the age of ridiculously rapid development that we have found ourselves in over the past few years. This is exactly why TensorFlow is dying
- amitport 4y ago
  I do not believe TensorFlow is anywhere close to dying.
- la_fayette 4y ago
  :) it still is impossible to bring a cnn rnn network in pytorch to mobile, which works fine with tflite...
  - antixk 4y ago
    Not sure if you have heard of Pytorch Mobile but it is very possible[0] [0] https://pytorch.org/tutorials/beginner/deeplabv3_on_ios.html https://pytorch.org/tutorials/beginner/deeplabv3_on_ios.html
    - perturbation 4y ago
      The big thing that PyTorch Mobile is lacking compared to TF Lite is on-device accelerator support (GPU/DSP/etc.) (there's experimental support for NNAPI https://pytorch.org/tutorials/prototype/nnapi_mobilenetv2.html https://pytorch.org/tutorials/prototype/nnapi_mobilenetv2.ht..., but this is a hack).
  - levesque 4y ago
    I don't think that's a problem for the vast majority of pytorch users
wging 4y ago
> Meta reportedly had more than 300 (?) people working on PyTorch, How much did this change after the big Meta layoffs? I think I know people who are no longer there, but I haven't talked to them about it yet.
- not2b 4y ago
  Because Meta open sourced it, and because PyTorch caught on, hopefully some of those laid off people can continue to work on it and also market themselves as PyTorch experts.
- PartiallyTyped 4y ago
  NB: PyTorch is now under the Linux Foundation. https://news.ycombinator.com/item?id=32810976 https://news.ycombinator.com/item?id=32810976
vikinghckr 4y ago
What I really find interesting here is that PyTorch, a library maintained by Facebook, is winning the marketshare and mindshare due to clean API, whereas Tensorflow, maintained by Google, is losing due to inferior API. In general, Google as a company emphasizes code quality and best practices far more than Facebook. But the story was reversed here.
- roenxi 4y ago
  Google's stereotype is the company that can handle and manage complexity - things like Kubernetes, indexing the internet, etc. They are clumsy at persuading people to use their products and have a patchy history of launching platforms that people want to use. Google+ and Google cloud vs AWS spring to mind, Kubernetes is a good platform but challenging to learn. Chrome is an unusual aberration where they did a great job. Android is maybe also a strong counterexample, not sure what the state of play was there. But whatever their code quality may be, luring in devs to their API has seen hits and misses.
  - wartijn_ 4y ago
    They did a pretty good job in persuading people to use search, adsense, YouTube, Gmail, Maps and a bunch of other products. Singling out a few couple of the less successful ones to claim a trillion dollar company is “clumsy at persuading people to use their products” seems like a pretty bad take.
    - roenxi 4y ago
      You'll note that all the examples you're citing were released more than a decade ago (search wasn't even this century!). And YouTube was an acquisition. They were basically the work of a different Google to the one that is walking around now. The stereotype of Google back then was very different. People would quote things like "don't be evil".
    - fsociety 4y ago
      The only products you listed which weren’t acquisitions are search and gmail. Search is exactly the problem Google was made to handle and Gmail.. well I don’t love gmail but it is a good enough product I guess?

t-vi 4y ago

> Another lesson is probably old but worth repeating: investment and professional polishing matters to open source projects While I'm not going to disagree that it matters, to my mind the main thing about early (0.x, 1.x) PyTorch's success was a clear vision (most visible to me from Soumith and Adam) that put the targeted users first (as in your productivity comment), awesome execution on it, and a creating a community where I, as an outside contributor with modes skill and no AI track record, and likely many others, felt welcome. Now, the many great people employed by Meta, but also other players like MS, NVidia, AMD, Intel, ... to work on PyTorch enabled all the things that make PyTorch 2.0 nicer and more broadly applicable than 1.0. (And to my mind it's quite a non-trivial accomplishment to enable other corporate players to join the party.)

SleekEagle 4y ago

Exactly, especially in the age of ridiculously rapid development that we have found ourselves in over the past few years. This is exactly why TensorFlow is dying

amitport 4y ago
I do not believe TensorFlow is anywhere close to dying.
la_fayette 4y ago
:) it still is impossible to bring a cnn rnn network in pytorch to mobile, which works fine with tflite...
- antixk 4y ago
  Not sure if you have heard of Pytorch Mobile but it is very possible[0] [0] https://pytorch.org/tutorials/beginner/deeplabv3_on_ios.html https://pytorch.org/tutorials/beginner/deeplabv3_on_ios.html
  - perturbation 4y ago
    The big thing that PyTorch Mobile is lacking compared to TF Lite is on-device accelerator support (GPU/DSP/etc.) (there's experimental support for NNAPI https://pytorch.org/tutorials/prototype/nnapi_mobilenetv2.html https://pytorch.org/tutorials/prototype/nnapi_mobilenetv2.ht..., but this is a hack).
- levesque 4y ago
  I don't think that's a problem for the vast majority of pytorch users

wging 4y ago

> Meta reportedly had more than 300 (?) people working on PyTorch, How much did this change after the big Meta layoffs? I think I know people who are no longer there, but I haven't talked to them about it yet.

not2b 4y ago
Because Meta open sourced it, and because PyTorch caught on, hopefully some of those laid off people can continue to work on it and also market themselves as PyTorch experts.
PartiallyTyped 4y ago
NB: PyTorch is now under the Linux Foundation. https://news.ycombinator.com/item?id=32810976 https://news.ycombinator.com/item?id=32810976

vikinghckr 4y ago

What I really find interesting here is that PyTorch, a library maintained by Facebook, is winning the marketshare and mindshare due to clean API, whereas Tensorflow, maintained by Google, is losing due to inferior API. In general, Google as a company emphasizes code quality and best practices far more than Facebook. But the story was reversed here.

roenxi 4y ago
Google's stereotype is the company that can handle and manage complexity - things like Kubernetes, indexing the internet, etc. They are clumsy at persuading people to use their products and have a patchy history of launching platforms that people want to use. Google+ and Google cloud vs AWS spring to mind, Kubernetes is a good platform but challenging to learn. Chrome is an unusual aberration where they did a great job. Android is maybe also a strong counterexample, not sure what the state of play was there. But whatever their code quality may be, luring in devs to their API has seen hits and misses.
- wartijn_ 4y ago
  They did a pretty good job in persuading people to use search, adsense, YouTube, Gmail, Maps and a bunch of other products. Singling out a few couple of the less successful ones to claim a trillion dollar company is “clumsy at persuading people to use their products” seems like a pretty bad take.
  - roenxi 4y ago
    You'll note that all the examples you're citing were released more than a decade ago (search wasn't even this century!). And YouTube was an acquisition. They were basically the work of a different Google to the one that is walking around now. The stereotype of Google back then was very different. People would quote things like "don't be evil".
  - fsociety 4y ago
    The only products you listed which weren’t acquisitions are search and gmail. Search is exactly the problem Google was made to handle and Gmail.. well I don’t love gmail but it is a good enough product I guess?