6 ms·
Inline Caching: Quickening
- pansa2 6y ago> improvements that could be made [...] Make a template interpreter like in the JVM. This will allow your specialized opcodes to directly make use of the call stack. What is a “template interpreter”? How does it differ from a normal bytecode interpreter?
- scaramanga 6y agoAs I understand it, template interpreters work by generating machine code corresponding to the virtual machine instructions and jumping to that machine code. As opposed to sort of a big switch statement that you'd get in a normal bytecode interpreter. A good intro to it here: https://programmersought.com/article/5521858566/ https://programmersought.com/article/5521858566/
- pansa2 6y agoThanks. Is there a performance advantage to the template approach - compared to writing the handler for each opcode directly in assembly language? Or is it just easier to implement?
- KMag 6y agoYou cut out the code between the opcodes that's used for figuring out what to do next. (Removing an indirect jump at the end of each opcode execution.)
- pansa2 6y agoThat sounds more like an advantage of a template compiler, not a template interpreter?
- chrisseaton 6y agoYes people in this thread are confusing the two. It reduces overhead because it directly refers to machine registers and the machine stack to pass information between instructions, and directly uses machine jumps, rather than going through a switch and method calls.
- pansa2 6y agoSounds like it’s similar to how LuaJIT works [0], using a machine-code handler for each VM opcode. AFAICT LuaJIT also uses a templating system to build these, except it generates them at compile-time instead of at run-time. I assume that’s easier than explicitly writing out each handler in assembly language. [0] https://www.reddit.com/r/programming/comments/badl2/luajit_2_beta_3_is_out_support_both_x32_x64/c0lrus0/ https://www.reddit.com/r/programming/comments/badl2/luajit_2...
- tekknolagi 6y agoYep, this is very similar.
- bjoli 6y agoA template JIT is a bit that just generates machine code that is exactly like the bytecode. That is in contrast to something like a tracing JIT that uses runtime information to generate more optimal machine code. One benefit of a template JIT is that the step to native compilation is a lot shorter than with a tracing JIT. Apart from being A LOT simpler. Debugging a tracing JIT is a special kind of hell for extra evil programmers.
- chrisseaton 6y agoThat's a 'template compiler' you're describing there, not a 'template interpreter'. OpenJDK doesn't include a 'template compiler'. Its lowest tier, C1, has a basic IR. OpenJDK does use a 'template interpreter'. https://github.com/openjdk/jdk/blob/master/src/hotspot/share/interpreter/templateInterpreter.cpp https://github.com/openjdk/jdk/blob/master/src/hotspot/share...
- bjoli 6y agoSorry about that. Thanks for putting it straight.
- deleted 6y ago[deleted]
- tekknolagi 6y agoI should be clearer and link to more resources. Check my PL resources page for example. But I also plan on writing a post about template interpreters soon!
- pansa2 6y agoIn Python, “Inline caching has been a huge success” [0]. I believe quickening has been proposed as part of a plan for further speed improvements [1]. [0] https://mobile.twitter.com/raymondh/status/1357478486647005187 https://mobile.twitter.com/raymondh/status/13574784866470051... [1] https://github.com/markshannon/faster-cpython/blob/master/plan.md https://github.com/markshannon/faster-cpython/blob/master/pl...
- chrisseaton 6y agoWere Python calls uncached before this month? That seems extraordinary and the world's lowest hanging fruit and I'm amazed nobody ever picked it. Dropbox had a whole team working on a Python JIT and didn't try that first?
- pansa2 6y agoI’m afraid I don’t know the details. However, this post seems relevant - someone who worked on the JIT at Dropbox describes how expensive function calls in CPython can be: https://blog.kevmod.com/2020/05/python-performance-its-not-just-the-interpreter/ https://blog.kevmod.com/2020/05/python-performance-its-not-j...
- jhayward 6y agoThe Python core developers have been working on caching in the interpreter for close to a decade[1]. They're very conservative, though, both about making the interpreter more complex as well as ensuring that nothing regresses in performance unless absolutely necessary to fix something broken. So I wouldn't exactly describe it as "low hanging fruit" [1] https://bugs.python.org/issue14757 https://bugs.python.org/issue14757
- vlovich123 6y agoTo my eye the follow up comments seem to imply it’s specifically for dict objects? > Methods already has their own fast path, so this is just for reading attributes from instances with __dict__. If __slots__ are defined, there is an addtional 35% improvement. But clearly it’s beyond just that: > Also, imports are now faster for modules that have type annotations. The annotations are initially loaded as strings and only evaluated later if needed. The only downside is that it easy to forget to make the relevant typing imports.