3 ms·
Neato! I'm surprised they went with a JIT compiler over a full-on compiler, but that might just be me not understanding: a) Compilers b) How a JIT compiler woul
by Seanny123 10y ago
Neato! I'm surprised they went with a JIT compiler over a full-on compiler, but that might just be me not understanding:
a) Compilers
b) How a JIT compiler would apply to this situation
My lab-mate Jan Gosmann recently did something similar for our spiking neural network software Nengo [1]. Although it isn't Deep Learning, it also builds a computational graph of operations. He ended up optimising the layout of the operations in memory to increase the efficiency of Numpy operations and reduce the amount of time spent in Python. He's in the process of writing a paper about it.
[1] https://github.com/nengo/nengo/pull/1035 https://github.com/nengo/nengo/pull/1035
- BenoitP 10y agoFrom my limited understanding: JITs can take into account the actual data being processed. And most importantly here, its size. Knowing the size will help with making chunks of work fit into L1, L2, L3 caches. Creating sensible SIMDs operations. Choosing what goes into a warp. Also, sometimes it is better to rematerialize a computation than having stored it; and the threshold to do it depends on the space and computing costs. Languages like Halide [1] let you hand-tune this threshold. I guess this is the kind of work XLA does here. [1] http://halide-lang.org/ http://halide-lang.org/
- emu 10y agoPrecisely. Just-in-time compilation allows the compiler to specialize the generated code for the shapes of the Tensors that appear at run time. This allows us to generate better code. XLA also has an experimental ahead-of-time mode, which we think will be particularly interesting for some production and mobile deployments. This is all work in progress though, and we're looking forward to getting the community involved.