7 ms·
JIT Compiling Code in 5μs
- glenjamin 1mo agopgrust sounds very interesting, but with the deep changes there’s no viable path to upstream it - is the end goal to be robust enough that it’ll get wide adoption?
- FiberBundle 1mo agoIs it really interesting though? It's essentially just vibe-coded by people who are unqualified for this kind of work. One of the authors claimed that what qualified them was having worked on a large-scale postgres cluster; they never actually worked on databases or compilers.
- genxy 1mo agoThey are going to lose their coding driver's license.
- roschdal 1mo agoJIT compilation is unsecure.
- dennis16384 1mo agoIt is the core of ClickHouse for example, for many years. Is it secure enough in your opinion?
- sebzim4500 1mo agoMaybe, but surely there are users who are willing to trust all users of their db instance.
- asdfsa32 1mo agoThe issue is that it restricts from locking-down and securing the system with Write xor Execute memory. So it has system wide implication. https://en.wikipedia.org/wiki/W%5EX https://en.wikipedia.org/wiki/W%5EX
- PhilipRoman 1mo agoW^X is typically per mapping, not per memory page and does not interfere with JIT compilation.
- asdfsa32 1mo agoSure, but it still means that the OS has to decide who is allowed to do it and to what extent. Sophisticated worms like Stuxnet would be much harder with strict W^X for example, since CVE-2010-2568 and the like would be much harder to execute.
- asdfsa32 1mo agoYou're entirely correct because JIT requires violating Write xor Execute security policy. This is the reason on iOS, it is limited to Apple shipped software. https://en.wikipedia.org/wiki/W%5EX https://en.wikipedia.org/wiki/W%5EX
- shakna 1mo agoThe wiki page mentions this is only a minor problem. Because everyone just writes, then switches and executes.
- codethief 1mo agoGrapheneOS heavily restricts JIT usage, too: > - Android Runtime Just-In-Time (JIT) compilation/profiling is fully disabled and replaced with full ahead-of-time (AOT) compilation. The only JIT compilation in the base OS is the V8 JavaScript JIT which is disabled by default for the Vanadium browser with per-site exception support. > - Dynamic code loading for both native code or Java/Kotlin classes is blocked for nearly the entire base OS. […] > - Dynamic code loading for both native code or Java/Kotlin classes can be disabled for user installed apps via 3 exploit protection toggles: […] https://grapheneos.org/features https://grapheneos.org/features
- norir 1mo agoAs someone who has written a jit compiler, I am puzzled by the claim that jitting requires write/execute permissions. When I have written a jit, I loaded some memory with read/write permissions using mmap. Once I filled in the generated code, I mprotected the region to read/execute before executing. The drawback to this approach is there can be some bloat because you can only mprotect at page granulariy so a jitted function that only takes say 10 bytes to represent would take up a full page in memory, but this is extreme and in practice, the overhead is unlikely to be worth worrying about.
- JackSlateur 1mo agoIn rust, is jit equivalent to an "unsafe" block ?
- pjmlp 1mo agoMachine code is insecure, we should all run interpreted code in a formally verified interpreter. Alternatively, only allow for the execution of cryptographly signed static linked binaries, this naturally includes the interpreter above.
- genxy 1mo agoIf we are sprinkling formal verification on things, we can sprinkle it on a JIT.
- bastawhiz 1mo agoBut then we'd have to admit that it's not the JIT that's a problem, it's the lack of guardrails and analysis features in the machine code interfaces that higher level languages expose!
- pjmlp 1mo agoOnly if it goes through verified bytecode, and the set of instructions is provable. The JIT must also only be allowed to call into specific code, controlled by the runtime, and nothing else.
- stevefan1999 1mo agoSo what, are you willing to go away from von-neumann architecture where instructions are data and data are instructions, i.e. the instruction-data hominocity that underpins JIT compilation? Are you willing to go to a pseudo-Harvard architecture where the ability of JIT compiling is soft locked by other means like VM or strong code authentication or policy protection, which is what Apple is doing. Fun fact: even Apple themselves have JIT. JavaScriptCore on iOS has JIT, it's just that the App Store policies forbid any application submissions with JIT or trying to mmap/mprotect an executable region. There used to be apps on TrollStore that runs JIT
- stevefan1999 1mo agoTypo: it is Homoiconicity, not hominocity. I typed too fast
- kllrnohj 1mo agoNo, it isn't. JITs don't grant capabilities abilities an equivalent interpreter doesn't already have. Allowing code execution allows code execution, that's it, that's the entirety of it.
- Ohentis 1mo ago[dead]
- MaxBarraclough 1mo agoThere's more to it than that though. Defects in JIT logic can lead to nasty low-level bugs. Plain old interpreters, especially if written in a safe language, are unlikely to have similar issues. This is important when the input code is untrusted. JIT bugs are a major source of browser vulnerabilities.
- kllrnohj 1mo agoJITs are an attack surface in that process. They are still restricted to things that process was already allowed to do, no matter how badly implemented the JIT is. In this example usage, one would hope that authentication already happened before the JIT processed the command. So an authorized user can attack themselves is the only realistic risk, which is hardly significant
- MaxBarraclough 1mo ago> JITs are an attack surface in that process There's plenty of scope for harm just within the process, even ignoring the possibility of escaping the process. In the case of a database server, essentially everything of value takes place within the database process (or processes). That process presumably has both access to the raw database data, and network access. We wouldn't want it sending data to an attacker's server. > one would hope that authentication already happened before the JIT processed the command We'd hope, yes, but SQL injection issues are still somewhat common. Also, an organisation might trust their DBMS to enforce permissions, and a JIT bug is the kind of thing that might allow non-permissioned data access. A DBMS should be hardened against malicious queries, just as a browser should be hardened against malicious JavaScript. In web browsers, the numbers show JIT compilers are a major cause of security issues. I don't know if there are hard numbers on JIT engines causing security issues in DBMSs though.
- glum64 1mo agoUhm, Common Lisp, where JIT is not only available but is also manageable: the programmer can decide what deserves to be compiled and what does not. Besides run time, JIT is available also when the code is compiled or loaded for execution (i.e., do you have a compilation or loading speed-up in mind? no problem, you can also compile that speed-up into native machine code, and so ad infinitum...).
- clbrmbr 1mo agois an xtensa lx7 (esp32-s3) target available that does not use llvm?
- glum64 1mo agoI am told there is http://www.ulisp.com/show?2AJI http://www.ulisp.com/show?2AJI Never used it myself; I cannot attest to the completeness of the implementation.
- fweimer 1mo agoNot in typical builds of SBCL: all code is compiled before evaluation.
- glum64 1mo agoIt depends on the implementation. CLISP compiles when it is told to.
- MaxBarraclough 1mo agoReminds me of the 2024 blog post Look ma, I wrote a new JIT compiler for PostgreSQL [0]. Both articles lament that Postgres's LLVM-based JIT [1] takes a while to generate code. > The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile. That's only true of writing a JIT from scratch. There's no rarity of JITs, it's just that LLVM (and other frameworks) are often used. Every major interpreter has a JIT compiler. PCRE2 has a JIT compiler. There are JIT frameworks out there with much faster code-generation than LLVM: Cranelift, GNU Lightning, Mir. I doubt they could do code-generation faster than a custom copy-and-patch JIT, but they'd be much faster than LLVM. [0] https://www.pinaraf.info/2024/03/look-ma-i-wrote-a-new-jit-compiler-for-postgresql/ https://www.pinaraf.info/2024/03/look-ma-i-wrote-a-new-jit-c... , discussed: https://news.ycombinator.com/item?id=39742916 https://news.ycombinator.com/item?id=39742916 [1] https://www.postgresql.org/docs/current/jit-reason.html https://www.postgresql.org/docs/current/jit-reason.html
- BoingBoomTschak 1mo agoA few other small and fast JITs: https://github.com/zherczeg/sljit https://github.com/zherczeg/sljit (used by libpcre), https://github.com/asmjit/asmjit https://github.com/asmjit/asmjit (RPCS3 and FBGEMM) and https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/ https://webkit.org/blog/5852/introducing-the-b3-jit-compiler... (only used by JSC in Webkit, I think)
- MaxBarraclough 1mo agoThanks, sljit looks somewhat similar to GNU lightning. On reflection I wonder if I overstated the widespread use of JIT and of JIT compiler frameworks. All the 'major' well-resourced high-profile JIT-based interpreters I can think of don't use an off-the-shelf JIT framework for their backend, which makes sense as they want to carefully tune the code-generation. OpenJDK, OpenJ9, .Net, V8, SpiderMonkey, JavaScriptCore. LuaJIT and Python's new JIT don't use one either, nor does the Linux kernel's BPF engine. The Guile Scheme interpreter uses a fork of the GNU Lightning JIT library. [0] Julia and (as mentioned) Postgres use LLVM for their JITs. I'm trying to think of other projects that use a JIT framework/library. Similarly, I can't think of many problem domains where it makes sense to use JIT. The ones that spring to mind are interpreters (of course), regex engines, and DBMSs. JIT can also help in high-performance computing, to tailor the code to the particular problem and the particular CPU. [1] I don't think there are many other contexts where it makes sense to use JIT though. JIT compilation brings its own drawbacks in portability (both between hardware platforms and operating systems), complexity, and perhaps cybersecurity, which might also limit its adoption, even if a good JIT framework could help with all three. [0] https://doc.guix.gnu.org/guile/latest/en/html_node/Just_002dIn_002dTime-Native-Code.html https://doc.guix.gnu.org/guile/latest/en/html_node/Just_002d... [1] https://www.intel.com/content/www/us/en/developer/articles/technical/onemkl-improved-small-matrix-performance-using-just-in-time-jit-code.html https://www.intel.com/content/www/us/en/developer/articles/t...
- hamilyon2 1mo agoIt uses copy-and-patch compilation to archive that
- varjag 1mo agoThere’s been a meme circulating about how AI doesn’t help because “code was never the hard part.” I think that’s true in some domains, but in others, writing the code absolutely was the hard part. JIT compilers are a great example of that.
- asdfsa32 1mo agoAnyone who thinks AI is good with writing code that is hard to write for the operator, not due to lack of basic software engineering know how but complexity of the domain, either has access to models beyond what is available to the public or is completely lost. I believe this because every time I use AI for domains that I consider myself above competent, if it is anything beyond UI components or a simple CRUD endpoints, I cringe at the quality of what it generates. This has made me to be extremely cautious of starting working in a new domain with AI if I want anything beyond throw away quick hacks or junk, shy of quick bug fixes perhaps.
- grebc 1mo agoThree quarters of my CS class at university could barely code and/or understand code. That’s not a joke. A lot went on to be programmers professionally. And judging by the quality of closed & open source code I witness daily those figures from university accurately depict people’s capabilities. Now that said, if you can’t really code then using AI will be a godsend to said individuals.
- asdfsa32 1mo agoFair, but I fear that now even more people who can't code will code, and code that is not any better than what people who could barely code write. Growing cabbages starting to look more and more interesting.
- varjag 1mo agoMixtral (a small local model) was churning out better code in fall 2023 that what an incompetent programmer would produce. Am sure others models could as well. It had severe limitations with tiny context and blind spots but still you'd never see it doing the kind of terrors a hack of a developer would do.
- mgaunard 1mo agoThe problem with the approach is that it's not real JIT-compilation, it's just assembly templates with basic substitutions. By not using LLVM, you're missing all the optimizations it does.
- mort96 1mo agoA non-optimizing compiler is a real compiler.
- mgaunard 1mo agoPeople do not call assemblers compilers, and yet, it's doing most of what the "JIT compiler" is doing.
- IshKebab 1mo agoThis absolutely is real JIT compilation. Copy and patch is a very well known JIT compilation technique.
- the-lazy-guy 1mo agoThis is absolutely a JIT-compiler. It compiles code into machine code. This is a surprisingly efficient way to get noticeable speedup relative to interpretation. Also it is much safer than proper optimising compiler. Say ebpf jit-compiler functions very similarly, because it is fast and _secure_ way to jit. (well, there's a bit of cheating because before emitting bpf bytecode it goes through gcc/clang pipeline). LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time.
- mort96 1mo agoTo back up the "surprisingly efficient way to get a speed-up" thing: I once wrote a toy compiler, without an optimizer and without even a register allocator (so all variables lived in stack memory). In my test benchmarks it was roughly 4x slower than Clang at -O3, IIRC. That's not exactly blazing fast for a low level C-like language, but it's not bad. It's infinitely faster than what I've ever gotten a toy interpreter to be.
- agnishom 1mo agoI recommend Russ Cox's articles on implementing a regex engine: https://swtch.com/~rsc/regexp/ https://swtch.com/~rsc/regexp/ It is very relevant
- ligarota 1mo agoTcc be like
- Gamer_S4lyer 1mo agoNice
- uygar 1mo ago[dead]
- malisper 1mo agoAuthor here. Let me know if you have any questions about the post or about pgrust.
- paidx 1mo ago[flagged]
- hnc3yfnu6f 1mo agoDidn't know that
- promptspheree 1mo ago[flagged]
- catlifeonmars 1mo agoThis is perhaps a little meta, but this was a pleasant read. It’s refreshing to read an article about using an LLM that doesn’t read like it was also written by that LLM. I might use this approach to generate the stencils for a JIT firewall I’ve been experimenting with. It also occurs to me that this could be used to generate eBPF byte code on the fly as well