9 ms·
When FFI Function Calls Beat Native C
- ars 8y agoTime to bring back self modifying code? A C program, that self modifies as it runs, to inline commonly called functions, or make direct jumps instead of indirect.
- bjpbakker 8y agoSo you mean a JIT compiler?
- chrisseaton 8y agoWe already have these - JIT compilers as the other person commented. You can even get them for C if you want.
- rkangel 8y agoIn general, you want to avoid self modifying code. You gain a major security benefit by ensuring that each page in the address space is "Writeable" OR "Executable" but not both. This is often abbreviated to W^X. Without W^X, many simple bugs can easily become bad exploits. You can inject some code into memory (using any program input) and then use the bug to cause it to be executed: immediate control. If W^X is in place, then the only code that could be executed by an exploit is code that is already there. Techniques like ROP allow this to be exploited still, but it's much harder and needs more of security hole to start with.
- nneonneo 8y agoThe PLT is mostly a necessary evil of two requirements: lazy-loaded symbols and W^X memory. The former is necessary to reduce binary load times, and the latter is necessary for security (RWX segments in memory are ripe for abuse in the event of a vulnerability). A JIT trades off very differently: slow startup ("warm up" period) and often creates RWX segments, but gains speed advantages as seen here. Of course, the GOT referred to by the PLT is itself is a bit of a security vulnerability, because it's usually a writable chunk of memory with a ton of frequently-called function pointers; corrupting it gives you relatively easy control over program flow. The new mitigation is "full RelRO", which resolves all GOT entries during binary load, then marks the GOT segment read-only. This gains in security, but trades off binary load time, and it is still going to be slower at runtime than the JIT. Full RelRO is usually off by default because of this added overhead. Modifying the code segment to include direct function references (either during binary load or later, during runtime) is also not a great approach because it means being unable to share the code segment between processes, which would increase memory pressure. We could do what Windows does, which is to have static randomization (randomization of binary addresses is performed once per boot, not once per process), enabling code to be "statically relocated" lazily per boot, but this carries its own complexities, and opens you up to different security vulnerabilities. There's no magic bullet here, unfortunately. There are just tradeoffs everywhere in many directions - between runtime speed, startup time, security and memory usage.
- ioquatix 8y agoStatic linking avoids this overhead right?
- richardwhiuk 8y agoIncreases memory usage, disk space, patch requirements and load time due to lack of shared library caching
- saagarjha 8y agoYes, at the cost of a bloated binary size.
- gaius 8y agoBut what if you link in just the functions you need, vs having the enormity of Library X on your disk that the package manager put there because it can’t know what you actually use/need?
- nneonneo 8y agoYes, a statically-linked binary will have all internal references resolved by the linker. If PIE is disabled, the addresses will all be fixed, which makes exploiting such a binary much easier. Glibc has only just added support for static PIE executables (in Feb of this year), and I'm not sure what the implementation is - whether it relocates all the references in the binary (increased memory usage), or uses a PLT/GOT approach to support relocations (increased runtime overhead).
- xyproto 8y agoHow important is code sharing between processes for a regular Linux desktop or server distros? Could JIT compiled applications be a sensible default in the future?
- exabrial 8y agoI didn't even know the plt existed... Fascinating. I'm curious if other JITd runtimes (like the jvm) offer the same optimizations
- dfox 8y agoTHis is essentially free optimization for every JITed runtime that is capable of inlining FFI calls because you get the real entry point from dlsym() and there is no sane way to get address of the .plt indirected entry point. On the other hand JVM AFAIK is not capable of inlining FFI calls (at the least because there is no API for direct FFI calls).
- electrum 8y agoThe JVM could do this, but there is a lot of other overhead for JNI calls (by virtue of the JNI API specification). Project Panama is working on improving native interoperability: http://openjdk.java.net/projects/panama/ http://openjdk.java.net/projects/panama/
- exabrial 8y agoNice! I didn't know this existed, but this is great.
- MaxBarraclough 8y agoI was under the impression that although C->Java is slow, Java->C calls have been fast for years, with the JIT generating full-speed calls - https://web.archive.org/web/20160304055443/http://nerds-central.blogspot.co.uk/2012/04/extreme-jni-performance.html https://web.archive.org/web/20160304055443/http://nerds-cent...
- aardvark179 8y agoIt's fast, but it depends on exactly what you're doing. There is always a bit of work managing the stack and converting between ABIs, and JNI is quite poor in performance when dealing with anything other than primitive types. The sort of thing Panama is adding is support for layouts of structures so that you can get results to and from C quickly, along with things like expressing vector algorithms in a way that allows good optimisation on different CPUs.
- jcranmer 8y agoThis is one of those things that sounds better than it really is. The timing used here in this post is ns per call, which somewhat obfuscates what's going on. A better metric would be clock cycles of overhead, because the differences here are only around 2-3 clock cycles. In other words, the potential savings is going to be drowned out by jitter caused by things like the processor deciding to service interrupts, or your hyperthread partner doing work. If the relevant code is so hot that the cost is showing up in profiles, then very likely, the fact that a function call is going through the PLT versus being directly called is not going to be the thing that makes the most difference: it's the fact that you're making the function call in the first place.
- ajross 8y agoWell, sure, but by the same measure and for the same reasons, "clock cycles" are not a deterministic measure of software performance, while "nanoseconds" are, by definition. The test can't easily control for cache effects (which are vastly bigger in aggregate than IRQ or SMT effects, FWIW), so it doesn't try and assumes everything averages out, and it does. The reason the effect seems bigger than it is is that for all but the most trivial leaf functions, PLT indirection just isn't a big deal. Never has been, isn't now, never will be. It's a cool trick that a JIT can skip that step, but there are bigger fish to fry.
- achamayou 8y agoAnd the most trivial cases will be inlined more often than not.
- nneonneo 8y agoFWIW, I get very different results from inside my Linux VM (MacBook Pro, 2.9 GHz Core i7): jit: 1.433856 ns/call plt: 1.700611 ns/call ind: 1.404397 ns/call The PLT is double-indirected (call->jmp->func), but the single-indirect call performs the same as the JIT. This backs up @tlb's note that modern x86 machines will predict indirect calls.
- vardump 8y agoWhich CPU you got? Lots of MacBook Pros have 2.9 GHz i7, ranging from 2012 to 2018 models.
- nneonneo 8y agoIt's a MacBookPro14,3, which makes it an i7-7820HQ according to everymac.
- glandium 8y agoSee https://news.ycombinator.com/item?id=17165535 https://news.ycombinator.com/item?id=17165535
- nneonneo 8y agoNow that I'm thinking about it a bit more - I recall that GCC 6.1 added "-fno-plt" for x64 targets, which replaces PLT calls with "call [func@got]" instead, eliminating the double-indirection. Unfortunately, I don't have a GCC 6.1 environment, so I'd be curious to see if someone can run the benchmark with "-fno-plt" and how it performs. A combination of that plus indirect call prediction should basically close the gap with the JIT approach.
- viraptor 8y agoFortunately "native C" is not one, unchangeable thing. If the plt overhead really is important in your situation, you can still compile either static, or with "--fno-plt".
- ndesaulniers 8y agoDamn, he beat me to the writeup. > The most surprising result of the benchmark is that LuaJIT’s FFI is substantially faster than C. https://news.ycombinator.com/threads?id=ndesaulniers https://news.ycombinator.com/threads?id=ndesaulniers
- gravypod 8y agoI'm still amazed that there is code in-lining across lineages. I saw that on the last post and I still can't imagine all of the edge cases that are being handled by that. Are there many other languages that do that other than LuaJIT?
- BeeOnRope 8y agoThe native code isn't being inlined into the compiled LUA code - it is still making a function call. It's just that the way it happens to do the call is faster than C calling a function in a shared object.
- cryptonector 8y agoRepeat after me: TFA is not about C nor Lua, it's about static vs. dynamic linking only.
- BeeOnRope 8y agoDid you actually read it? It compares three cases: dynamic linking, calls through a function pointer and calls where the address to call is compiled directly into an immediate call instruction using a small JIT intended to emulate the LuaJIT case. The benchmark didn't even include a static linking case! In general, however, I expect the static linking case to perform like the JIT case since both will almost always use a call with 32-bit relative immediate jump. I was a bit confused about why the article author chose to write a JIT rather than just use static linking, since I think they achieve the same thing here.
- nicoburns 8y agoRust is planning to enable inlining C functions in the near future. I believe this should work in both directions, and works because they are both compiled using LLVM.
- steveklabnik 8y agohttps://github.com/rust-lang/rust/pull/50000 https://github.com/rust-lang/rust/pull/50000 is the first bit, https://github.com/rust-lang/rust/issues/49879 https://github.com/rust-lang/rust/issues/49879 is the tracking issue.
- wvenable 8y agoThis reminds me of working with C++ and the Pebble SDK. The Pebble builds ELF binaries but doesn't have an ELF loader. It also uses position independent code. The SDK uses objtools to strip the code out of the ELF and the Pebble OS just copies the code into memory and jumps to it. In C, this works fine. But in C++, gcc generates the GOT for virtual method calls but there is no ELF loader to fill it. I accidentally found out if my code contains a destructor that makes an assignment to a static variable then no GOT is generated and virtual method calls work for the entire project! I'm not sure the logic behind that but this is the code that is needed: class SampleObject { public: //! Destroys the underlying Pebble window virtual ~SampleObject() { // The assignment to a static variable makes the virtual methods work for subclasses // I have no idea why! dummy() = 0; } private: // Dummy static variable needed for virutal method pointers to work (see destructor) inline static int& dummy() { static int dummy = 0; return dummy; } }
- newnewpdro 8y agoPLT and GOT are quite orthogonal to the C programming language, these are implementation details of dynamic linking and shared libraries. Back in the 90s we would statically link performance-sensitive executables to avoid this overhead. It didn't require changing the language.
- FrozenVoid 8y agoA rule of thumb:If you see "faster than C" claims its either poor C or they use different code path/algorithm thats not equivalent to C code. A FFI that is faster than native, isn't FFI but some other mechanism that more direct - in this case bypassing the address of the function resolution. 'static inline' is of course even faster. The only C overhead i recall that is legit is libc startup time with glibc, and you can use faster libc alternatives.
- gaius 8y agoUnless it is FORTRAN obviously
- Noughmad 8y agoFORTRAN is only faster than C because it doesn't allow aliasing. Use the `restrict` keyword in C and you should get FORTRAN-like performance.
- ArneBab 8y agoFortran can give you much better speed at comparable code-complexity, as long as your problem is one which FORTRAN is optimized for, i.e. multidimensional array manipulation or numerical work.
- baruchthescribe 8y agoAnother rule of thumb: if you see "faster than C" and the article mentions LuaJIT, then it's correct.
- cryptonector 8y agoTFA is not really about C either. Nor is it about Lua. It's about static vs. dynamic linking and nothing more. It's just irritating and click-baity.
- BeeOnRope 8y agoHow is the example in this article and the article is based on not equivalent to the C functionality? Yes, the JIT is able to take advantage of runtime code generation to embed the final loaded address in the compiled assembly, which is something C _cannot do_ or at least does not do in its default configuration to support PIC, symbol interposition and other features. JITs can have all the same support, but because of runtime code generation (which comes with its own costs) the final call-site can be more efficient. So it's a reasonable rule of thumb, but it has plenty of exceptions. Other examples include JITs creating code optimized to the runtime-observed data distribution and JITs efficiently using specific hardware features of the current hardware rather than statically compiling in lowest-common-denominator or trying to use runtime dispatch.
- jorangreef 8y agoIt would be great if V8 could follow LuaJIT in this.
- lukego 8y agoLuaJIT is awesome. It's just amazing how much information is available to the JIT. I've been using it for more than five years now and I still can't believe that such a high-level dynamic language actually is competitive with C for system programming. That Mike guy knows a thing or two I reckon.
- cryptonector 8y agoUh, whatever. This isn't about LuaJIT, nor any JIT, nor C, nor any high-level programming language FFI. ELF != C C does not imply ELF ELF does not imply C This is about entirely and only dynamic linking vs. static linking. Selling it as anything else is bunk and click-bait. And yes, dlsym(3) should return the address of the PLT, not the direct address of the function: so that dlsym(foo, "bar") == bar (when you can link with whatever object provides "bar").
- abacate 8y agoAny micro performance gains from a function call will become rounding errors compared to the extra cost of any arithmetic operation on a dynamically-typed language. It may be an interesting article for describing how JIT and some late-optimization techniques can produce optimized code given the right optimization preconditions, but I fail to see how the title is supposed to describe the contents of the article. Last, but not least, performance-sensitive C code tends to be statically linked.
- stmw 8y agoEspecially timely as Spectre/Meltdown mitigations put indirect calls and branch predictions under more performance pressure -- nice reminder of how these things work.
- wwarner 8y agoimmutability ftw