6 ms·

Tiny JITs for a Faster FFI

316 points by hahahacorn 2y ago

pestatije 2y ago
FFI - Foreign Function Interface, or how to call C from Ruby
- tonetegeatinst 2y ago
  The totally safe and sane approach is to write C code that gets passed data via the command line during execution, then vomits results to the command line or just into a memory page. Then just execute the c program with your flags or data in the terminal using ruby and viola, ruby can run C code.
  - grandempire 2y ago
    This. I think many people do not understand Unix processes and don’t realizing how rare it is to need bindings, ffi, and many libraries. How many programs have an https client in them because they need to make one request and didn’t know they could use curl?
    - nirvdrum 2y ago
      Can you please elaborate on this because I'm struggling to follow your suggestion. Shelling out to psql every time I want to run an SQL query is going to be prohibitively slow. It seems to me you'd need bindings in almost the exact same cases you'd use a shared library if you were writing in C and that's really all bindings are anyway -- a bridge between the VM and a native library.
      - grandempire 2y ago
        Spawning a process isn't the right tool for ALL X-language communication. But sometimes it is - and the bias tends to be to overlook these opportunities. When you are comfortable using libraries, you make more libraries. When you know how to use programs, you more often make programs. > Shelling out to psql I would recommend using a Postgres connection library, because that's how Postgres is designed. Note that ongoing communication can still work with a multi-process stdin/stdout design. This is how email protocols work. So someone could design a SQL client that works this way. I have absolutely written batch import scripts which simply spawn psql for every query, with great results. > in almost the exact same cases you'd use a shared library That's the thing. Libraries are an entangling relationship (literally in your binary). Programs in contrast have a clean, modular interface. So for example you can choose to load the imagemagick library, or you can spawn imagemagick. Which one is better depends, but most often you don't need the library. Here is a list of examples I have seen solved with a library that were completely unnecessary: - identify the format of an image. - zip some data - post usage analytics to a web server at startup - diff two pieces of data - convert math symbols to images - convert x format to y I have even seen discourse online that suggests that if you are serious about AI your web stack needs to be in python - as if you can't spawn a process for an AI job.
        nirvdrum 2y ago
        > Spawning a process isn't the right tool for ALL X-language communication. But sometimes it is I'm with you here. > ...many people do not understand Unix processes and don’t realizing how rare it is to need bindings, ffi, and many libraries But, this is a much stronger claim. I can't tell if you're making a meta point or addressing something in the Ruby ecosystem. I mentioned database library bindings because that's far and away the most common usage in the Ruby ecosystem, particularly because of its frequent adoption for web applications. The author is advocating for not using native code at all if you can avoid it. Keep as much code in Ruby as you can and let the JIT optimize it. But, if you do need bindings, it'd be great if you didn't have to write a native extension. There are a lot of ways to shoot yourself in the foot and they complicate the build pipeline. However, historically, FFI has been much slower than writing a native extension. The point of this post is to explore a way to speed up FFI in the cases where you need it. It needs to be taken on faith that the author is doing this work because he either has performance sensitive code or needs to work with a library. Spawning a new process in that case is going to be slower than any of the options explored in the post. Writing and distributing a C/C++/Zig/Rust/Go application as part of a Rails app is a big hammer to swing and complicates deployments (a big part of the reason to move away from native extensions). It's possible the author is just complicating things unnecessarily, but he's demonstrated a clear mastery of the technologies involved so I'm willing to give him the benefit of the doubt. A frequent critique of Ruby is that it's slow. Spawning processes for operations on the hot path isn't going to help that perception. I agree there are cases where shelling out makes sense. E.g., spawning out to use ImageMagick has proven to be a better approach than using bindings when I want to make image thumbnails. But, those are typically handled in asynchronous jobs. I'm all for anything we can do to speed up the hot path and it's remarkable how much performance was easily picked up.
        grandempire 2y ago
        You interpreted my comment as an attack on the author. If they have a special case they want to optimize FFI for this seems like a great way to do it. > Spawning processes for operations on the hot path isn't going to help that perception. Yes it will. Because then you do the number crunching in C/Java/Rust etc instead of Ruby. And the OS will do multi-core scheduling for free. This is playing to Ruby’s strength as a high level orchestrator. I think you’re vastly overestimating the fork/exec overhead on Linux. If your hot path is this hot you better not be calling Ruby functions. > Writing and distributing a C/C++/Zig/Rust/Go application as part of a Rails app is a big hammer to swing You’re already doing it. Do you have a database? Nginx? Me cached? Imagemagick? Do you use containers? Consider the alternative: is the ideal world the one where all software is available as a ruby package?
  - fomine3 2y ago
    It's slow
poisonta 2y ago
I can sense why it didn’t go to tenderlovemaking.com
- tenderlove 2y ago
  I think tenderworks wrote this post.
internetter 2y ago
"write as much Ruby as possible, especially because YJIT can optimize Ruby code but not C code" I feel like I'm not getting something. Isn't ruby a pretty slow language? If I was dipping into native I'd want to do as much in native as possible.
- kevingadd 2y ago
  When dealing with a managed language that has a JIT or AOT compiler it's often ideal to write lots of stuff in the managed language, because that enables inlining and other optimizations that aren't possible when calling into C. This is sometimes referred to as "self-hosting", and browsers do it a lot by moving things into privileged JavaScript that might normally have been written in C/C++. Surprisingly large amounts of the standard library end up being not written in native code.
  - internetter 2y ago
    Oh, I am indeed surprised! I guess I always assumed that most of the JavaScript standard library was written in C++
    - achierius 2y ago
      Well, most all of the compiler, runtime, allocator, garbage collector, object model, etc, are indeed written in C++ And so are many special operations (eg crypto functions, array sorts, walking the stack) But specifically with regards to library functions, like the other commentator said, losing out on in lining sucks, and crossing between JS and Native code can be pretty expensive, so even with things like sorting an array it can be better to do it in js to avoid the overhead... Eg esp in cases where you can provide a callback as your comparator, which is js, and thus you have to cross back into js for every element So it's a balancing game, and the various engines have gone back and forth on which functions are implemented in which language over time
    - jitl 2y ago
      Much of the standard library in v8 is written in Torque, a custom language. https://v8.dev/docs/torque https://v8.dev/docs/torque Example file for array.find(…): https://github.com/v8/v8/blob/5fe0aa3bc79c0a9d3ad546b79211f07105f09585/src/builtins/array-find.tq#L120 https://github.com/v8/v8/blob/5fe0aa3bc79c0a9d3ad546b79211f0...
shortrounddev2 2y ago
Does ruby have its equivalent to typescript, with type annotations? The language sounds interesting but I tend not to give dynamically typed languages the time of day
- kevingadd 2y ago
  There's https://sorbet.org/ https://sorbet.org/ but it's not clear whether it has much adoption.
  - zem 2y ago
    I continue to think it was a big mistake not to add syntactic support for type annotations into the base language. python did this right; annotations are not enforced by the interpreter, but are accessible both by external tools as part of the AST and bytecode, and by the running program via introspection, so tools and libraries can do all sorts of interesting things with them. having to add annotations in a separate header file is simply too high friction to get widespread adoption.
    - Lammy 2y ago
      IMHO (and I don't expect most people to agree but please be tolerant of my opinion!) annotations are annoying busywork that clutter my code and exist just to make people feel smart for “““doing correctness”””. The only check I find useful is nil or not-nil, and any halfway-well-designed interface should make it impossible for some unexpected object type to end up in the wrong place anyway. For anything less than halfway-well-defined, you have bigger issues than a lack of type annotation. edit: I am quite fond of `case ::Ractor::receive; when SomeClass then …; when SomeOtherClass then …; end` as the main pattern for my Ractors though :)
      - zem 2y ago
        as your codebase and number of collaborators get larger, it's super useful to have the type checker be able to tell you "hey, you said your function arg could be a time or an int, but you are calling time-specific methods on it" or conversely "the function you are calling says it accepts time objects but you are passing it an int" also once you get into jit compilation you can do some nice optimisations if you can treat a variable type as statically known rather than dynamic. and finally, even if you're not writing python at scale it can be very nice to use the type annotations to document your function parameters.
chris12321 2y ago
Between Rails At Scale and byroot's blogs, it's currently a fantastic time to be interested in in-depth discussions around Ruby internals and performance! And with all the recent improvements in Ruby and Rails, it's a great time to be a Rubyist in general!
- jupp0r 2y ago
  Is it? To me it seems like Ruby is declining [1]. It's still popular for a specific niche of applications, but to me it seems like it's well past its days of glory. Recent improvements are nice, but is a JIT really that exciting technologically in 2025? [1]: https://www.tiobe.com/tiobe-index/ruby/ https://www.tiobe.com/tiobe-index/ruby/
  - chris12321 2y ago
    Ruby will probably never again be the most popular language in the world, and it doesn't need to be for the people who enjoy it to be excited about the recent improvements in performance, documentation, tooling, ecosystem, and community.
    - faizshah 2y ago
      I think ruby can get popular again with the sort of contrarian things Rails is doing like helping developers exit Cloud. There isn’t really a much more productive web dev setup than Rails + your favorite LLM tool. Will take time to earn Gen Z back to Rails though and away from Python/TS or Go/Rust.
      - jimmaswell 2y ago
        My impression is that a Rails app is an unmaintainable dynamically-typed ball of mud that might give you the fast upfront development to get to a market or get funded but will quickly fall apart at scale, e.g. Twitter fail whale. And Ruby is too full of "magic" that quickly makes it too hard to tell what's going on or accidentally make something grossly inefficient if you don't understand the magic, which defeats the point of the convenience. Is this perception outdated, and if so what changed?
nialv7 2y ago
isn't this exactly what libffi does?
- tenderlove 2y ago
  libffi can't know how to unwrap Ruby types (since it doesn't know what Ruby is). The advantage presented in this post is that the code for type unboxing is basically "cached" in the generated machine code based on the information the user passes when calling `attach_function`.
- almostgotcaught 2y ago
  you know i thought i knew what libffi was doing (i thought it was playing tricks with GOT or something like that) but i think you're right https://github.com/libffi/libffi/blob/master/src/tramp.c https://github.com/libffi/libffi/blob/master/src/tramp.c
- dzaima 2y ago
  libffi doesn't JIT for FFI calls; and it still requires you to lay out argument values yourself, i.e. for a string argument you'd still need to write code that converts a Ruby string object to a C string pointer. And libffi is rather slow. (the tramp.c linked in a sibling comment is for "reverse-FFI", i.e. exposing some dynamic custom operation as a function pointer; and its JITting there amounts to a total of 3 instructions to call into precompiled code)
- kazinator 2y ago
  libffi is slow; it doesn't JIT as far as I know. In libffi you built up descriptor objects for functions. These are run-time data structures which indicate the arguments and return value types. When making a FFI call, you must pass in an array of pointers to the values you want to pass, and the descriptor. Inside libffi there is likely a loop which walks the loop of values, while at the same time traversing the descriptor, and places those values onto the stack in the right way according to the type indicated in the descriptor. When the function is done, it then pulls out the return according to its type. It's probably switching on type for all these pieces. Even if the libffi call mechanism were JITted, the preparation of the argument array for it would still be slow. It's less direct than a FFI jit that directly accesses the arguments without going through an intermediate array. FFI JIT code will directly take the argument values, convert them from the Ruby (or whatever) type to the C type, and stick it into the right place on the stack or register, and do that with inline code for each value. Then call the function, and convert the return value to the Ruby type. Basically as if you wrote extension code by hand: // Pseudo-code RubyValue *__Generated_Jit_Code(RubyValue *arg1, RubyValue *arg2) { return CStringToRuby(__targetCFunction(RubyToCString(arg1), RubyToCInt(arg2)); } If there is type inference, the conversion code can skip type checks. If we have assurance that arg1 is a Ruby string, we can use an unsafe, faster version of the RubyToCString function. The JIT code doesn't have to reflect over anything other than at worst the Ruby types. It doesn't have to have any array or list related to the arguments. It knows which C types are being converted to and form, and that is hard-coded: there is no data structure describing the C side that has to be walked at run0-time.
haberman 2y ago
> Rather than calling out to a 3rd party library, could we just JIT the code required to call the external function? I am pretty sure this is the basis of the LuaJIT FFI: https://luajit.org/ext_ffi.html https://luajit.org/ext_ffi.html I think LuaJIT's FFI is very fast for this reason.
aidenn0 2y ago
Why does this need to be JIT compiled? If it could be written in C, then it certainly could just be compiled at load time, no?
- nirvdrum 2y ago
  If what could be written in C? The FFI library allows for dynamic binding of library methods for execution from Ruby without the need to write a native extension. That's a huge productivity boost and makes for code that can be shared across CRuby, JRuby, and TruffleRuby. I suppose if you could statically determine all of the bindings at boot up you could write a stub and insert into the method table. But, that still would happen at runtime, making it JIT. And it wouldn't be able to adapt to the types flowing through the system, so it'd have to be conservative in what it accepts or what it optimizes, which is what libffi already does today. The AOT approach is to write a native extension.
  - aidenn0 2y ago
    By "it" I meant this part from TFA: > you should write a native extension with a very very limited API where most work is done in Ruby. Any native code would be a very thin wrapper around the function we actually want to call that just converts Ruby types in to the types required by the native function. I think our main disagreement is your assertion that any compilation at runtime qualifiees as JIT. I consider JIT to be dynamic compilation (and possibly recompilation) of a running program, not merely anything that generates machine code at runtime.
brigandish 2y ago
It's an aside, but > Now, usually I steer clear of FFI, and to be honest the reason is simply that it doesn’t provide the same performance as a native extension. I usually avoid it, or in particular, gems that use it, because compilation can be such a pain. I've found it easier to build it myself and cut out the middleman of Rubygems/bundler.
cchianel 2y ago
I had to deal with a lot of FFI to enable a Java Constraint Solver (Timefold) to call functions defined in CPython. In my experience, most of the performance problems from FFI come from using proxies to communicate between the host and foreign language. A direct FFI call using JNI or the new foreign interface is fast, and has roughly the same speed as calling a Java method directly. Alas, the CPython and Java garbage collectors do not play nice, and require black magic in order to keep them in sync. On the other hand, using proxies (such as in JPype or GraalPy) cause a significant performance overhead, since the parameters and return values need to be converted, and might cause additional FFI calls (in the other direction). The fun thing is if you pass a CPython object to Java, Java has a proxy to the CPython object. And if you pass that proxy back to CPython, a proxy to that proxy is created instead of unwrapping it. The result: JPype proxies are 1402% slower than calling CPython directly using FFI, and GraalPy proxies are 453% slower than calling CPython directly using FFI. What I ultimately end up doing is translating CPython bytecode into Java bytecode, and generating Java data structures corresponding to the CPython classes used. As a result, I got a 100x speedup compared to using proxies. (Side note: if you are thinking about translating/reading CPython bytecode, don't; it is highly unstable, poorly documented, and its VM has several quirks that make it hard to map directly to other bytecodes). For more details, you can see my blog post on the subject: https://timefold.ai/blog/java-vs-python-speed https://timefold.ai/blog/java-vs-python-speed
- LinXitoW 2y ago
  Speaking from zero experience, the FFI stories of both Python and Java to C seems much better. Wouldn't going connecting them via a little C bridge a general solution?
  - cchianel 2y ago
    JNI/the new Foreign FFI communicate with CPython via CPython's C API. The primary issue is getting the garbage collectors to work with each other. The Java solver works by repeatedly calling user defined functions when calculating the score. As a result: - The Java side needs to store opaque Python pointers which may have no references on the CPython side. - The CPython side need to store generated proxies for some Java objects (the result of constraint collectors, which are basically aggregations of a solution's data). Solving runs a long time, typically at least a hour (although you can modify how long it runs for). If we don't free memory (by releasing the opaque Python Pointer return values), we will quickly run out of memory after a couple of minutes. The only way to free memory on the Java side is to close the arena holding the opaque Python pointer. However, when that arena is closed, its memory is zeroed out to prevent use-after-free. As a result, if CPython haven't garbage collected that pointer yet, it will cause a segmentation fault on the next CPython garbage collection cycle. JPype (a CPython -> Java bridge) does dark magic to link the JVM's and CPython's garbage collector, but has performance issues when calling a CPython function inside a Java function, since its proxies have to do a lot of work. Even GraalPy, where Python is ran inside a JVM, has performance issues when Python calls Java code which calls Python code.
eay_dev 2y ago
I've been using Ruby more than 10 years, and seeing its development in these days is very exciting. I hope
IshKebab 2y ago
> Even in those cases, I encourage people to write as much Ruby as possible, especially because YJIT can optimize Ruby code but not C code. But the C code is still going to be waaay faster than the Ruby code even with YJIT. That seems like an odd reason to avoid C. (I think there are other good reasons though.)
- Alifatisk 2y ago
  > the C code is still going to be waaay faster than the Ruby code even with YJIT. I can't find it but I remember seeing a talk where they showed examples of Ruby + YJIT hitting the same speed and in some cases a bit more than C. The downside was though that it required to some warmup time.
  - IshKebab 2y ago
    I find that hard to believe. I've heard claims JIT can beat C for years, but they usually involve highly artificial microbenchmarks (like Fibonacci) and even for a high performance JITed language like Java it ends up not beating C. There's no way YJIT will. The YJIT website itself only claims it is around twice as fast as Ruby, which means it is still slower than a flat earth snail. The benchmarks game has YJIT and it's somewhere between PHP and Python. Very slow.
    - Alifatisk 2y ago
      I’ll see if I can find the presentation again on Youtube
  - igouy 2y ago
    Maybe "Ruby Outperforms C" was bait for "Outperforming a C extension" which is about the relative cost of calling C from Ruby? https://railsatscale.com/2023-08-29-ruby-outperforms-c/ https://railsatscale.com/2023-08-29-ruby-outperforms-c/
cpursley 2y ago
Can anyone still make a case to start a new project in Rails in 2025 when there is Elixir LiveView? I enjoy Ruby but activerecord is a mess and the language is slow and lacks real time functionality.
- arrowsmith 2y ago
  Rails has more mindshare, it's easier to hire for, there are more tutorials etc to help you when you get stuck, and Ruby has a more mature ecosystem of libraries/plugins than Elixir has. I'd still pick Phoenix over Rails any day of the week, but if I had to make the case for Rails, that would be it.
  - cpursley 2y ago
    That's fair. Are there any particular libs for Elixir that you found missing?
    - arrowsmith 2y ago
      I can't think of anything specific that was a huge problem. But when you're integrating with a third-party tool (e.g. for analytics, error reporting, email delivery, anything really), it's very common to see that they provide a Ruby SDK but not an Elixir SDK. Or if there's an Elixir SDK, it's something unofficial (and possibly unmaintained) from a third party, while the Ruby SDK is something in-house that has official support. Most of the time these SDKs are just thin wrappers around their JSON API, so it's easy to build your own thing in Elixir anyway. And LLMs make it really easy. But it's something I've experienced pretty often.
- adenta 2y ago
  Is anyone using LiveView in production?
  - arrowsmith 2y ago
    Someone asked that question on r/elixir just the other day: https://old.reddit.com/r/elixir/comments/1inebf4/what_saas_startup_you_build_with_phoenix/ https://old.reddit.com/r/elixir/comments/1inebf4/what_saas_s...
evacchi 2y ago
somewhat related, this library uses the JVMCI (JVM Compiler Interface) to generate arm64/amd64 code on the fly to call native libraries without JNI https://github.com/apangin/nalim https://github.com/apangin/nalim

316 points by hahahacorn 2y ago

pestatije 2y ago

FFI - Foreign Function Interface, or how to call C from Ruby

tonetegeatinst 2y ago
The totally safe and sane approach is to write C code that gets passed data via the command line during execution, then vomits results to the command line or just into a memory page. Then just execute the c program with your flags or data in the terminal using ruby and viola, ruby can run C code.
- grandempire 2y ago
  This. I think many people do not understand Unix processes and don’t realizing how rare it is to need bindings, ffi, and many libraries. How many programs have an https client in them because they need to make one request and didn’t know they could use curl?
  - nirvdrum 2y ago
    Can you please elaborate on this because I'm struggling to follow your suggestion. Shelling out to psql every time I want to run an SQL query is going to be prohibitively slow. It seems to me you'd need bindings in almost the exact same cases you'd use a shared library if you were writing in C and that's really all bindings are anyway -- a bridge between the VM and a native library.
    - grandempire 2y ago
      Spawning a process isn't the right tool for ALL X-language communication. But sometimes it is - and the bias tends to be to overlook these opportunities. When you are comfortable using libraries, you make more libraries. When you know how to use programs, you more often make programs. > Shelling out to psql I would recommend using a Postgres connection library, because that's how Postgres is designed. Note that ongoing communication can still work with a multi-process stdin/stdout design. This is how email protocols work. So someone could design a SQL client that works this way. I have absolutely written batch import scripts which simply spawn psql for every query, with great results. > in almost the exact same cases you'd use a shared library That's the thing. Libraries are an entangling relationship (literally in your binary). Programs in contrast have a clean, modular interface. So for example you can choose to load the imagemagick library, or you can spawn imagemagick. Which one is better depends, but most often you don't need the library. Here is a list of examples I have seen solved with a library that were completely unnecessary: - identify the format of an image. - zip some data - post usage analytics to a web server at startup - diff two pieces of data - convert math symbols to images - convert x format to y I have even seen discourse online that suggests that if you are serious about AI your web stack needs to be in python - as if you can't spawn a process for an AI job.
      - nirvdrum 2y ago
        > Spawning a process isn't the right tool for ALL X-language communication. But sometimes it is I'm with you here. > ...many people do not understand Unix processes and don’t realizing how rare it is to need bindings, ffi, and many libraries But, this is a much stronger claim. I can't tell if you're making a meta point or addressing something in the Ruby ecosystem. I mentioned database library bindings because that's far and away the most common usage in the Ruby ecosystem, particularly because of its frequent adoption for web applications. The author is advocating for not using native code at all if you can avoid it. Keep as much code in Ruby as you can and let the JIT optimize it. But, if you do need bindings, it'd be great if you didn't have to write a native extension. There are a lot of ways to shoot yourself in the foot and they complicate the build pipeline. However, historically, FFI has been much slower than writing a native extension. The point of this post is to explore a way to speed up FFI in the cases where you need it. It needs to be taken on faith that the author is doing this work because he either has performance sensitive code or needs to work with a library. Spawning a new process in that case is going to be slower than any of the options explored in the post. Writing and distributing a C/C++/Zig/Rust/Go application as part of a Rails app is a big hammer to swing and complicates deployments (a big part of the reason to move away from native extensions). It's possible the author is just complicating things unnecessarily, but he's demonstrated a clear mastery of the technologies involved so I'm willing to give him the benefit of the doubt. A frequent critique of Ruby is that it's slow. Spawning processes for operations on the hot path isn't going to help that perception. I agree there are cases where shelling out makes sense. E.g., spawning out to use ImageMagick has proven to be a better approach than using bindings when I want to make image thumbnails. But, those are typically handled in asynchronous jobs. I'm all for anything we can do to speed up the hot path and it's remarkable how much performance was easily picked up.
        grandempire 2y ago
        You interpreted my comment as an attack on the author. If they have a special case they want to optimize FFI for this seems like a great way to do it. > Spawning processes for operations on the hot path isn't going to help that perception. Yes it will. Because then you do the number crunching in C/Java/Rust etc instead of Ruby. And the OS will do multi-core scheduling for free. This is playing to Ruby’s strength as a high level orchestrator. I think you’re vastly overestimating the fork/exec overhead on Linux. If your hot path is this hot you better not be calling Ruby functions. > Writing and distributing a C/C++/Zig/Rust/Go application as part of a Rails app is a big hammer to swing You’re already doing it. Do you have a database? Nginx? Me cached? Imagemagick? Do you use containers? Consider the alternative: is the ideal world the one where all software is available as a ruby package?
- fomine3 2y ago
  It's slow

poisonta 2y ago

I can sense why it didn’t go to tenderlovemaking.com

tenderlove 2y ago
I think tenderworks wrote this post.

internetter 2y ago

"write as much Ruby as possible, especially because YJIT can optimize Ruby code but not C code" I feel like I'm not getting something. Isn't ruby a pretty slow language? If I was dipping into native I'd want to do as much in native as possible.

kevingadd 2y ago
When dealing with a managed language that has a JIT or AOT compiler it's often ideal to write lots of stuff in the managed language, because that enables inlining and other optimizations that aren't possible when calling into C. This is sometimes referred to as "self-hosting", and browsers do it a lot by moving things into privileged JavaScript that might normally have been written in C/C++. Surprisingly large amounts of the standard library end up being not written in native code.
- internetter 2y ago
  Oh, I am indeed surprised! I guess I always assumed that most of the JavaScript standard library was written in C++
  - achierius 2y ago
    Well, most all of the compiler, runtime, allocator, garbage collector, object model, etc, are indeed written in C++ And so are many special operations (eg crypto functions, array sorts, walking the stack) But specifically with regards to library functions, like the other commentator said, losing out on in lining sucks, and crossing between JS and Native code can be pretty expensive, so even with things like sorting an array it can be better to do it in js to avoid the overhead... Eg esp in cases where you can provide a callback as your comparator, which is js, and thus you have to cross back into js for every element So it's a balancing game, and the various engines have gone back and forth on which functions are implemented in which language over time
  - jitl 2y ago
    Much of the standard library in v8 is written in Torque, a custom language. https://v8.dev/docs/torque https://v8.dev/docs/torque Example file for array.find(…): https://github.com/v8/v8/blob/5fe0aa3bc79c0a9d3ad546b79211f07105f09585/src/builtins/array-find.tq#L120 https://github.com/v8/v8/blob/5fe0aa3bc79c0a9d3ad546b79211f0...

shortrounddev2 2y ago

Does ruby have its equivalent to typescript, with type annotations? The language sounds interesting but I tend not to give dynamically typed languages the time of day

kevingadd 2y ago
There's https://sorbet.org/ https://sorbet.org/ but it's not clear whether it has much adoption.
- zem 2y ago
  I continue to think it was a big mistake not to add syntactic support for type annotations into the base language. python did this right; annotations are not enforced by the interpreter, but are accessible both by external tools as part of the AST and bytecode, and by the running program via introspection, so tools and libraries can do all sorts of interesting things with them. having to add annotations in a separate header file is simply too high friction to get widespread adoption.
  - Lammy 2y ago
    IMHO (and I don't expect most people to agree but please be tolerant of my opinion!) annotations are annoying busywork that clutter my code and exist just to make people feel smart for “““doing correctness”””. The only check I find useful is nil or not-nil, and any halfway-well-designed interface should make it impossible for some unexpected object type to end up in the wrong place anyway. For anything less than halfway-well-defined, you have bigger issues than a lack of type annotation. edit: I am quite fond of `case ::Ractor::receive; when SomeClass then …; when SomeOtherClass then …; end` as the main pattern for my Ractors though :)
    - zem 2y ago
      as your codebase and number of collaborators get larger, it's super useful to have the type checker be able to tell you "hey, you said your function arg could be a time or an int, but you are calling time-specific methods on it" or conversely "the function you are calling says it accepts time objects but you are passing it an int" also once you get into jit compilation you can do some nice optimisations if you can treat a variable type as statically known rather than dynamic. and finally, even if you're not writing python at scale it can be very nice to use the type annotations to document your function parameters.

chris12321 2y ago

Between Rails At Scale and byroot's blogs, it's currently a fantastic time to be interested in in-depth discussions around Ruby internals and performance! And with all the recent improvements in Ruby and Rails, it's a great time to be a Rubyist in general!

jupp0r 2y ago
Is it? To me it seems like Ruby is declining [1]. It's still popular for a specific niche of applications, but to me it seems like it's well past its days of glory. Recent improvements are nice, but is a JIT really that exciting technologically in 2025? [1]: https://www.tiobe.com/tiobe-index/ruby/ https://www.tiobe.com/tiobe-index/ruby/
- chris12321 2y ago
  Ruby will probably never again be the most popular language in the world, and it doesn't need to be for the people who enjoy it to be excited about the recent improvements in performance, documentation, tooling, ecosystem, and community.
  - faizshah 2y ago
    I think ruby can get popular again with the sort of contrarian things Rails is doing like helping developers exit Cloud. There isn’t really a much more productive web dev setup than Rails + your favorite LLM tool. Will take time to earn Gen Z back to Rails though and away from Python/TS or Go/Rust.
    - jimmaswell 2y ago
      My impression is that a Rails app is an unmaintainable dynamically-typed ball of mud that might give you the fast upfront development to get to a market or get funded but will quickly fall apart at scale, e.g. Twitter fail whale. And Ruby is too full of "magic" that quickly makes it too hard to tell what's going on or accidentally make something grossly inefficient if you don't understand the magic, which defeats the point of the convenience. Is this perception outdated, and if so what changed?

nialv7 2y ago

isn't this exactly what libffi does?

tenderlove 2y ago
libffi can't know how to unwrap Ruby types (since it doesn't know what Ruby is). The advantage presented in this post is that the code for type unboxing is basically "cached" in the generated machine code based on the information the user passes when calling `attach_function`.
almostgotcaught 2y ago
you know i thought i knew what libffi was doing (i thought it was playing tricks with GOT or something like that) but i think you're right https://github.com/libffi/libffi/blob/master/src/tramp.c https://github.com/libffi/libffi/blob/master/src/tramp.c
dzaima 2y ago
libffi doesn't JIT for FFI calls; and it still requires you to lay out argument values yourself, i.e. for a string argument you'd still need to write code that converts a Ruby string object to a C string pointer. And libffi is rather slow. (the tramp.c linked in a sibling comment is for "reverse-FFI", i.e. exposing some dynamic custom operation as a function pointer; and its JITting there amounts to a total of 3 instructions to call into precompiled code)
kazinator 2y ago
libffi is slow; it doesn't JIT as far as I know. In libffi you built up descriptor objects for functions. These are run-time data structures which indicate the arguments and return value types. When making a FFI call, you must pass in an array of pointers to the values you want to pass, and the descriptor. Inside libffi there is likely a loop which walks the loop of values, while at the same time traversing the descriptor, and places those values onto the stack in the right way according to the type indicated in the descriptor. When the function is done, it then pulls out the return according to its type. It's probably switching on type for all these pieces. Even if the libffi call mechanism were JITted, the preparation of the argument array for it would still be slow. It's less direct than a FFI jit that directly accesses the arguments without going through an intermediate array. FFI JIT code will directly take the argument values, convert them from the Ruby (or whatever) type to the C type, and stick it into the right place on the stack or register, and do that with inline code for each value. Then call the function, and convert the return value to the Ruby type. Basically as if you wrote extension code by hand: // Pseudo-code RubyValue *__Generated_Jit_Code(RubyValue *arg1, RubyValue *arg2) { return CStringToRuby(__targetCFunction(RubyToCString(arg1), RubyToCInt(arg2)); } If there is type inference, the conversion code can skip type checks. If we have assurance that arg1 is a Ruby string, we can use an unsafe, faster version of the RubyToCString function. The JIT code doesn't have to reflect over anything other than at worst the Ruby types. It doesn't have to have any array or list related to the arguments. It knows which C types are being converted to and form, and that is hard-coded: there is no data structure describing the C side that has to be walked at run0-time.

haberman 2y ago

> Rather than calling out to a 3rd party library, could we just JIT the code required to call the external function? I am pretty sure this is the basis of the LuaJIT FFI: https://luajit.org/ext_ffi.html https://luajit.org/ext_ffi.html I think LuaJIT's FFI is very fast for this reason.

aidenn0 2y ago

Why does this need to be JIT compiled? If it could be written in C, then it certainly could just be compiled at load time, no?

nirvdrum 2y ago
If what could be written in C? The FFI library allows for dynamic binding of library methods for execution from Ruby without the need to write a native extension. That's a huge productivity boost and makes for code that can be shared across CRuby, JRuby, and TruffleRuby. I suppose if you could statically determine all of the bindings at boot up you could write a stub and insert into the method table. But, that still would happen at runtime, making it JIT. And it wouldn't be able to adapt to the types flowing through the system, so it'd have to be conservative in what it accepts or what it optimizes, which is what libffi already does today. The AOT approach is to write a native extension.
- aidenn0 2y ago
  By "it" I meant this part from TFA: > you should write a native extension with a very very limited API where most work is done in Ruby. Any native code would be a very thin wrapper around the function we actually want to call that just converts Ruby types in to the types required by the native function. I think our main disagreement is your assertion that any compilation at runtime qualifiees as JIT. I consider JIT to be dynamic compilation (and possibly recompilation) of a running program, not merely anything that generates machine code at runtime.

brigandish 2y ago

It's an aside, but > Now, usually I steer clear of FFI, and to be honest the reason is simply that it doesn’t provide the same performance as a native extension. I usually avoid it, or in particular, gems that use it, because compilation can be such a pain. I've found it easier to build it myself and cut out the middleman of Rubygems/bundler.

cchianel 2y ago

I had to deal with a lot of FFI to enable a Java Constraint Solver (Timefold) to call functions defined in CPython. In my experience, most of the performance problems from FFI come from using proxies to communicate between the host and foreign language. A direct FFI call using JNI or the new foreign interface is fast, and has roughly the same speed as calling a Java method directly. Alas, the CPython and Java garbage collectors do not play nice, and require black magic in order to keep them in sync. On the other hand, using proxies (such as in JPype or GraalPy) cause a significant performance overhead, since the parameters and return values need to be converted, and might cause additional FFI calls (in the other direction). The fun thing is if you pass a CPython object to Java, Java has a proxy to the CPython object. And if you pass that proxy back to CPython, a proxy to that proxy is created instead of unwrapping it. The result: JPype proxies are 1402% slower than calling CPython directly using FFI, and GraalPy proxies are 453% slower than calling CPython directly using FFI. What I ultimately end up doing is translating CPython bytecode into Java bytecode, and generating Java data structures corresponding to the CPython classes used. As a result, I got a 100x speedup compared to using proxies. (Side note: if you are thinking about translating/reading CPython bytecode, don't; it is highly unstable, poorly documented, and its VM has several quirks that make it hard to map directly to other bytecodes). For more details, you can see my blog post on the subject: https://timefold.ai/blog/java-vs-python-speed https://timefold.ai/blog/java-vs-python-speed

LinXitoW 2y ago
Speaking from zero experience, the FFI stories of both Python and Java to C seems much better. Wouldn't going connecting them via a little C bridge a general solution?
- cchianel 2y ago
  JNI/the new Foreign FFI communicate with CPython via CPython's C API. The primary issue is getting the garbage collectors to work with each other. The Java solver works by repeatedly calling user defined functions when calculating the score. As a result: - The Java side needs to store opaque Python pointers which may have no references on the CPython side. - The CPython side need to store generated proxies for some Java objects (the result of constraint collectors, which are basically aggregations of a solution's data). Solving runs a long time, typically at least a hour (although you can modify how long it runs for). If we don't free memory (by releasing the opaque Python Pointer return values), we will quickly run out of memory after a couple of minutes. The only way to free memory on the Java side is to close the arena holding the opaque Python pointer. However, when that arena is closed, its memory is zeroed out to prevent use-after-free. As a result, if CPython haven't garbage collected that pointer yet, it will cause a segmentation fault on the next CPython garbage collection cycle. JPype (a CPython -> Java bridge) does dark magic to link the JVM's and CPython's garbage collector, but has performance issues when calling a CPython function inside a Java function, since its proxies have to do a lot of work. Even GraalPy, where Python is ran inside a JVM, has performance issues when Python calls Java code which calls Python code.

eay_dev 2y ago

I've been using Ruby more than 10 years, and seeing its development in these days is very exciting. I hope

IshKebab 2y ago

> Even in those cases, I encourage people to write as much Ruby as possible, especially because YJIT can optimize Ruby code but not C code. But the C code is still going to be waaay faster than the Ruby code even with YJIT. That seems like an odd reason to avoid C. (I think there are other good reasons though.)

Alifatisk 2y ago
> the C code is still going to be waaay faster than the Ruby code even with YJIT. I can't find it but I remember seeing a talk where they showed examples of Ruby + YJIT hitting the same speed and in some cases a bit more than C. The downside was though that it required to some warmup time.
- IshKebab 2y ago
  I find that hard to believe. I've heard claims JIT can beat C for years, but they usually involve highly artificial microbenchmarks (like Fibonacci) and even for a high performance JITed language like Java it ends up not beating C. There's no way YJIT will. The YJIT website itself only claims it is around twice as fast as Ruby, which means it is still slower than a flat earth snail. The benchmarks game has YJIT and it's somewhere between PHP and Python. Very slow.
  - Alifatisk 2y ago
    I’ll see if I can find the presentation again on Youtube
- igouy 2y ago
  Maybe "Ruby Outperforms C" was bait for "Outperforming a C extension" which is about the relative cost of calling C from Ruby? https://railsatscale.com/2023-08-29-ruby-outperforms-c/ https://railsatscale.com/2023-08-29-ruby-outperforms-c/

cpursley 2y ago

Can anyone still make a case to start a new project in Rails in 2025 when there is Elixir LiveView? I enjoy Ruby but activerecord is a mess and the language is slow and lacks real time functionality.

arrowsmith 2y ago
Rails has more mindshare, it's easier to hire for, there are more tutorials etc to help you when you get stuck, and Ruby has a more mature ecosystem of libraries/plugins than Elixir has. I'd still pick Phoenix over Rails any day of the week, but if I had to make the case for Rails, that would be it.
- cpursley 2y ago
  That's fair. Are there any particular libs for Elixir that you found missing?
  - arrowsmith 2y ago
    I can't think of anything specific that was a huge problem. But when you're integrating with a third-party tool (e.g. for analytics, error reporting, email delivery, anything really), it's very common to see that they provide a Ruby SDK but not an Elixir SDK. Or if there's an Elixir SDK, it's something unofficial (and possibly unmaintained) from a third party, while the Ruby SDK is something in-house that has official support. Most of the time these SDKs are just thin wrappers around their JSON API, so it's easy to build your own thing in Elixir anyway. And LLMs make it really easy. But it's something I've experienced pretty often.
adenta 2y ago
Is anyone using LiveView in production?
- arrowsmith 2y ago
  Someone asked that question on r/elixir just the other day: https://old.reddit.com/r/elixir/comments/1inebf4/what_saas_startup_you_build_with_phoenix/ https://old.reddit.com/r/elixir/comments/1inebf4/what_saas_s...

evacchi 2y ago

somewhat related, this library uses the JVMCI (JVM Compiler Interface) to generate arm64/amd64 code on the fly to call native libraries without JNI https://github.com/apangin/nalim https://github.com/apangin/nalim