6 ms·
XLA: linear algebra library for TensorFlow
- theCricketer 10y agoChris Leary, a compiler engineer at Google gave a talk about XLA at the recent Tensorflow Dev Summit: https://www.youtube.com/watch?v=kAOanJczHA0 https://www.youtube.com/watch?v=kAOanJczHA0
- probdist 10y agoWhy does this support JIT but not AOT for NVIDIA GPUs?
- learyg 10y agoAOT for GPUs is doable. Do you have a killer use case? For CPU, mobile code footprint reduction was the driving force.
- puzzle 10y agoNot having to ship a toolchain (nvcc, gpucc or whatever equivalent linked as a library)?
- posterboy 10y agowhich from dev POV means having to ship a tool(chain) (for the AoT stuff)!
- learyg 10y agoIt's gpucc, so it builds from LLVM when you enable XLA in the TF configure step. Trying to understand: you don't want to ship a compiler library on principle, or is it some kind of product requirement, or ? There's lots of cool work to be done in the compiler space so use cases help to prioritize. :-) Thanks!
- Marat_Dukhan 10y ago>>> Softmax can be implemented as a composition of primitive TensorFlow ops (exponent, reduction, elementwise division, etc.): softmax = exp(logits) / reduce_sum(exp(logits), dim) No, it can not be implemented this way, it is numerically unstable, and will produce NaNs if any input is greater than ~88.7. Luckily, it is also not how its implemented in Tensorflow: https://github.com/tensorflow/tensorflow/blob/2c8d0dca978a246f54c506aae4587dbce5d3bcf0/tensorflow/core/kernels/softmax_op_functor.h#L43 https://github.com/tensorflow/tensorflow/blob/2c8d0dca978a24... For a clean (and more efficient) C version of this algorithm, take a look at NNPACK reference implementation: https://github.com/Maratyszcza/NNPACK/blob/master/src/ref/softmax-output.c#L30 https://github.com/Maratyszcza/NNPACK/blob/master/src/ref/so...
- learyg 10y agoThe XLA and vanilla TF variants appear to be the same here: https://github.com/tensorflow/tensorflow/blob/2c8d0dca978a246f54c506aae4587dbce5d3bcf0/tensorflow/compiler/tf2xla/kernels/softmax_op.cc#L50 https://github.com/tensorflow/tensorflow/blob/2c8d0dca978a24... https://github.com/tensorflow/tensorflow/blob/2c8d0dca978a246f54c506aae4587dbce5d3bcf0/tensorflow/core/kernels/softmax_op_functor.h#L67 https://github.com/tensorflow/tensorflow/blob/2c8d0dca978a24... Chalk it up to poetic license? ;-)
- acmj 10y agoIn the nnpack implementation, the same exponential (i.e. expf) is computed twice for each element, which is a waste of time. A faster implementation should save each expf result to output[sample][channel] first, compute the sum and then rescale output[sample][channel] by the sum.
- Marat_Dukhan 10y agoIf you have an efficient vectorized implementation of expf (NNPACK does), softmax is a memory/cache bandwidth-bound kernel, and storing data is less efficient than recomputing.
- acmj 10y ago
- visarga 10y agoIt would seem Torch/PyTorch are faster than TF. TF uses static optimizations on the computation graph while Torch has a dynamic computation graph. Logically, static optimizations should be faster because they know the data size beforehand. So, why is TF slower?
- pmalynin 10y agoTensorflow is getting dynamic jit optimization too. I think part of the reason that some dynamic optimizations might perform better is that the results of the optimization can be cached for most other batches and they can specialize to utilize batch/shape/input specific properties
- killjoywashere 10y agoI'm kind of getting the sense that TF is presently being optimized for its own massive development: that is, the engineers are yanking out chunks of code and replacing them, quickly, and at varying scales.
- snnn 10y agoTF has runtime cost based optimization, which is inside Session.run()
- gcr 10y agoA nitpick, but an important one: Only Pytorch uses a dynamic computation graph. Torch doesn't have any concept of a computation graph.
- jakekovoor 10y agoThank you OP, this is really helpful. :) If you need to install TensorFlow on Windows 10 you can follow this http://saintlad.com/install-tensorflow-on-windows/ http://saintlad.com/install-tensorflow-on-windows/
- alok-g 10y agoI would like to see the guide for the GPU version. Is it coming anytime soon? :-) Thanks.
- ska 10y agoanaconda has gpu and non gpu packages, if that helps.
- dr_zoidberg 10y agoEasier than that, I got Python 3.6 64bits and headed to Christoph Gohlke's amazing Python Wheels repository, downloaded everything I needed (NumPy, SciPy, OpenCV 3.2, matplotlib, etc + TensorFlow) and then it was just a matter of: pip3 install [filename] ...until everything was installed. I gotta buy that man more than a few beers for all the time he's saved me over the years. Bonus points: got the wheels with Intel MKL and OpenCV+contrib. [0] http://www.lfd.uci.edu/~gohlke/pythonlibs/ http://www.lfd.uci.edu/~gohlke/pythonlibs/
- shoshin23 10y agoI've been looking around in a few places but I can't find a way to use XLA to compile tensorflow models for mobile devices. Is there a tutorial/blogpost by google(or anyone for that matter) talking about it? Thanks!
- learyg 10y agoDid you see the "using tfcompile" section of the docs? https://www.tensorflow.org/versions/master/experimental/xla/tfcompile#using_tfcompile https://www.tensorflow.org/versions/master/experimental/xla/... If you're looking for more detailed information that's missing from the docs, please do file a Github issue about it. Thanks!
- ndesaulniers 10y agoEven if you're not interested in machine learning or ai, XLA and particularly it's Python bindings are a great and easy way to do GPGPU programming.