4 ms·
In my experience (computer vision, deep learning) PyTorch is substantially faster as well, especially in data augmentation where it’s not just a thin layer over
by 0xbear 9y ago
In my experience (computer vision, deep learning) PyTorch is substantially faster as well, especially in data augmentation where it’s not just a thin layer over cudnn.
That said, you’re right. There’s no way I’d deploy it to production.
- cjalmeida 9y agoWhy wouldn't you deploy it in production - on server side at least?
- 0xbear 9y agoBecause TF lets you run _just_ the model, with no Python stuff at all. For us there is no “server side”, our stuff is deployed on prem. The fewer variables there are the easier our lives are going to be. Additionally, PyTorch download page warns you point blank that it’s an early version of the software and that you should “expect some adventures”. Adventures are fine for research, but inadvisable in production IMO.
- cjalmeida 9y agoI understand the appeal, but IMO wrapping the Python stuff in a docker container is fairly simple. Making it run at scale using something like kubernetes is more advanced stuff but still within good devops practices. Unless, of course if you want to run on exotic HW, like TPUs. But that's an issue for Googlish scales. Edit: BTW, if on premise you mean Windows client software your point is totally valid, Python would suck for this.
- puzzle 9y agoDepending on how much extra logic you add around the model serving library, then you are exposing your source code, too. Maybe you care, maybe not. Re: Windows, there's no TF serving for that, so TF is no better than other Python frameworks.
- nicodjimenez 9y agoNonsense, Pytorch is great in production. I use it for Mathpix (mathpix.com) which processes 20 million images per month.
- boulos 9y agoDon't take this the wrong way, but 20M images per month is only about 20 qps or so (20M / 3600 / 730 => 7.6 qps, but that's unrealistically even). That's pretty manageable even on a single box, depending on the network and the size of the box. When people say they're concerned about a system in production, they often mean when needing to roll it out to many machines due to higher scalability requirements. Disclosure: I work at Google, but not TensorFlow.
- edanm 9y agoThat's a fair point, but just as a counterpoint, for plenty of people, 20M per month is a lot of traffic, and it's good and valid to hear about experiences of people putting PyTorch into production with this scale.
- agibsonccc 9y ago"Production" tends to be an overloaded term. Arguments like this tend to lead to "My production is bigger than your production!" which is frankly silly. You could mean large scale, real time, "small batch job with online lookups from a CRUD database",.. Let's just admit it's use case specific and move on.
- 0xbear 9y agoJust one instance of our product has to process 39 million inferences per month (number is dictated by frame rate), with predictable latency, 24x7, on both Windows and Linux.
- foo101 9y agoAbout 8 images per second? How does that prove that Pytorch is great in production? I am not saying that Pytorch is bad in production but I fail to see how your metric of 8 images per second proves anything.
- Marat_Dukhan 9y agoHave you looked at ONNX? It is a neural network exchange format that, in particular, lets you deploy PyTorch models in production via Caffe2. Here is a tutorial: http://pytorch.org/docs/master/onnx.html http://pytorch.org/docs/master/onnx.html Disclaimer: I work on Caffe2 team (not on ONNX, though)
- ffriend 9y agoIf I want to export my own computational graph to ONNX, what is the first place I should look at? Do you know about any documentation or reference implementation of the format?
- Marat_Dukhan 9y agoReference specification and validator are hosted in https://github.com/onnx/onnx https://github.com/onnx/onnx
- agibsonccc 9y agoWe are taking a pretty hard look at ONNX right now. It doesn't support a lot of ops yet. I would add that it's very early for supporting anything "real" yet.
- syllogism 9y agoDoes the ONNX thing actually work? I'm super skeptical of these interchanges, because it seems very difficult to avoid train/test skew. Any difference in detail between the two implementations is a potential problem. I can imagine different order of operations putting out some values by 0.1%, causing 1% eventual loss of accuracy.
- Marat_Dukhan 9y agoIt should, if you find that converted model doesn't work, it is a bug, please file it with PyTorch and/or Caffe2. Implementation details of course differ between frameworks, but luckily neural networks are very robust to noise. In my experience, changes like using Winograd/Fourier for convolutions, or even running the whole thing in FP16 do not result in noticeable artifacts, and these are among the biggest differences you could have between frameworks.
- walrus 9y ago> There’s no way I’d deploy it to production. I've seen this stated by a few people, but never justified. What are some reasons for not deploying PyTorch in production?