9 ms·
PyTorch 1.0 is out
- dspoka 8y agoIs there some sort of pytorch 1.0 migration guide or does anyone know if there is any breaking from .41 to 1.0 ?
- shafte 8y agoBreaking changes are highlighted here: https://github.com/pytorch/pytorch/releases#breaking-changes https://github.com/pytorch/pytorch/releases#breaking-changes
- smhx 8y agoin the release notes, there's a section called "Breaking Changes". There are no major breaking changes from 0.4.1, and all your code should largely work as-is.
- amelius 8y ago> The JIT is a set of compiler tools for bridging the gap between research in PyTorch and production. It allows for the creation of models that can run without a dependency on the Python interpreter and which can be optimized more aggressively. Using program annotations existing models can be transformed into Torch Script, a subset of Python that PyTorch can run directly. Isn't python bytecode simple enough that it can be run anywhere?
- skybrian 8y agoAlternative implementations of Python don't seem very easy to write, so I'm guessing the answer is no.
- yorwba 8y agoPython is not very difficult to implement [1], the hard part is making it fast. That's because every property access involves a lot of magic behind the scenes, like __getattribute__, __getattr__ and the method resolution order. And because Python is dynamically typed, that dispatch logic can't be compiled away but needs to execute every time. (PyPy's JIT can probably speed it up, but still needs deoptimization checks in case types change.) [1] see https://github.com/nedbat/byterun https://github.com/nedbat/byterun for a Python implementation
- rspeer 8y agoPython bytecode is a pre-processed form of instructions to a Python interpreter, such as getting the function with a specified name out of a Python library and calling it with certain Python objects as arguments. The point here is to run PyTorch code without having a Python interpreter, and without having to run slow Python code.
- levan_ts 8y agoThere was a huge problem with converting weight normalization module (torch.utils.weight_norm) and with forward and pre-forward hooks generally, so I had to rewrite the full model. Hope they've improved it in the stable release
- tbenst 8y agoThe new JIT is very interesting. Anyone know if this is for inference only or also for training?
- smhx 8y agotraining and inference.
- dekimir 8y agoCan you actually train a torch::jit::script::Module? I couldn't figure out how to feed its parameter tensors to a torch::optim::OptimizerBase constructor...
- amelius 8y agoI wonder what is "just in time" about that compiler, does anyone know? If you want to compile your python code so that it can run in C++, then you'll have to statically compile, no?
- deleted 8y ago[deleted]
- Rafuino 8y agoIs this the version combining Caffe2 and PyTorch into one framework? Is LMDB continuing as the default data load/store method?
- thelastidiot 8y agoAccording to this [0] blog, it it: PyTorch 1.0 takes the modular, production-oriented capabilities from Caffe2 and ONNX and combines them with PyTorch's existing flexible, research-focused design to provide a fast, seamless path from research prototyping to production deployment for a broad range of AI projects. [0] https://developers.facebook.com/blog/post/2018/05/02/announcing-pytorch-1.0-for-research-production/ https://developers.facebook.com/blog/post/2018/05/02/announc...
- htfy96 8y agoWhat surprised me most is the elegance of C++ API. Compared to its equivalence in Python, the C++ version is almost the same if we discard the "auto" keyword [0]. As mentioned in the doc, they put user-friendliness over micro-optimizations, which also proves the expressiveness of modern C++ (at least when they want to prioritize user-friendliness!) [0]: https://pytorch.org/cppdocs/frontend.html#end-to-end-example https://pytorch.org/cppdocs/frontend.html#end-to-end-example
- perturbation 8y agoI'm hoping that this results in a nice, high-level API for https://github.com/fragcolor-xyz/nimtorch https://github.com/fragcolor-xyz/nimtorch as well, which AFAIK has been wrapping the low-level Aten API. I've been keeping my eye on that project, and been really excited about it.
- usgroup 8y agoi picked up c++ for some gpu stuff with the arrayfire api... felt the same. firstly, modern c++ takes no time to learn if you come from java / c/ c# / etc. secondly, things like operator overloading and type inference make for pretty seamless apis. E.g. want to add matrices? auto C = A + B. A lot of things suck (closures, generator functions, first order functions all suck in c++), but oh my does it all run fast when you get it working; plus wrapping it with Rcpp or Lua is easy as pie.
- 8y ago
- l3robot 8y agoTL;DR - New JIT feature that lets you run your model without python. It now seems trivial to load a pytorch model in C++ - New distributed computation package. Major redesign. - C++ frontend - New torch hub feature to load models from github easily
- make3 8y agoI love PyTorch, but my experience with jits embedded in Python (eg. Numba) has been everything but simple, nevermind trivial. I'll really have to try it to believe it.
- mlthoughts2018 8y agoI’ve had the opposite experience with numba in production. It works almost flawlessly, very easy to reason about the generated code and inspect annotations, easy to debug.
- alexbw 8y agoCheck out github.com/google/jax, it’s NumPy on the GPU with automatic differentiation, JIT and autobatching.
- mlthoughts2018 8y agoThat’s very cool. Numba and Cython work extremely well with virtually no overhead or extra effort on my part, so jax doesn’t seem like it would buy me much for most of my work. But I can imagine a lot of projects where jax woukd be useful, and I plan to keep current on best practices for it.
- make3 8y agoDo you interact with numpy or other compiled numerical packages? this is where it usually breaks for me. The thing is that I use numpy, scipy, keras, tensorflow, etc. in literally every project, making numba not too useful
- wpietri 8y agoIn case anybody else was wondering, since this isn't in the fine article: "PyTorch is a Python package that provides two high-level features: * Tensor computation (like NumPy) with strong GPU acceleration * Deep neural networks built on a tape-based autograd system You can reuse your favorite Python packages such as NumPy, SciPy and Cython to extend PyTorch when needed."
- gnulinux 8y agoML noob hobbyist here. Would you use PyTorch for models not involving deep neural networks too or is it just good for that. Say if I use linear models (like least squares etc) or use custom algorithms (integer linear programming, optimization, or something else...) but need very fast linear algebra support is PyTorch a good lib? I'm a C kinda guy so I usually use blas, lapack etc or numpy+pandas+sklearn in python. Would PyTorch give a "complete" enough feel or would I just use it only for nn and use other libraries for other things?
- p1esk 8y agoPyTorch uses CuBLAS [1] under the hood, among other libraries, so basic linear algebra ops should be fast. You might also look at CuPy [2], especially if you like NumPy. [1] https://developer.nvidia.com/cublas https://developer.nvidia.com/cublas [2] https://cupy.chainer.org/ https://cupy.chainer.org/
- openbasic 8y agoWhat would be a good book and project to get started with this? Object recognition? Product recommendations?
- ipsum2 8y agoThe best resource I've found are Udacity's Jupyter notebooks: https://github.com/udacity/deep-learning-v2-pytorch https://github.com/udacity/deep-learning-v2-pytorch. Start with intro-to-pytorch.
- dangom 8y agoI would suggest going through the Fast.AI course [1]. It's an excellent course to learn more about DL in general, and some of torch API. The downside is that the course material was produced when PyTorch was still at 0.3, so some of the API has changed since then. [1] https://course.fast.ai/ https://course.fast.ai/
- jph00 8y agoA new version for PyTorch v1 will be released next month, FYI.
- jeffreysmith 8y agoThere are several good books written or in development, but if I had to pick one right now, I'd point you to this one: https://www.manning.com/books/deep-learning-with-pytorch https://www.manning.com/books/deep-learning-with-pytorch
- openbasic 8y agoThank you! :)
- nafizh 8y agoReally grateful to the FAIR team for Pytorch. I use deep learning for computational biology. Pytorch lets me focus on the problem rather than nitpicking with the framework (looking at you tensorflow) to make something work.
- pesenti 8y agoLink to blog post: https://code.fb.com/ai-research/pytorch-developer-ecosystem-expands-1-0-stable-release/ https://code.fb.com/ai-research/pytorch-developer-ecosystem-...
- ritoune 8y agoI've been using PyTorch, and the PyTorch 1.0 pre-release for a while now. I adore it but don't really want to write C++ backends in production. Anyone want to start working on Golang bindings for C++ PyTorch?
- metahost 8y agoCurious, can SWIG[1] help automate the process? [1]: http://www.swig.org/ http://www.swig.org/
- Solar19 8y agoAgreed. I've always thought, however, that D would be the perfect language for a new Python implementation.
- pilooch 8y agoIf someone from fair is around, what does this mean for caffe2 ? Is the new torch c++ API a form a replacement for caffe?
- punnerud 8y agoFeels good to see torch-1.0.0 when I do: "python -m pip install torch torchvision --upgrade"
- dekimir 8y agoHm, the Mac version of LibTorch is suddenly unavailable!? [0] I swear it was available for download until a few days ago... [0] https://pytorch.org/get-started/locally/ https://pytorch.org/get-started/locally/
- smhx 8y agowe are fixing this shortly. it was an oversight. Edit: fixed links should be going live in a few mins, via: https://github.com/pytorch/pytorch.github.io/pull/141 https://github.com/pytorch/pytorch.github.io/pull/141