3 ms·

> But most of the speedup right now is still coming from rewriting C into Ruby. Quick glance, this statement seems backwards - shouldn't C always be faster? or

by uticus 1y ago

> But most of the speedup right now is still coming from rewriting C into Ruby. Quick glance, this statement seems backwards - shouldn't C always be faster? or maybe i'm misunderstanding how the JIT truly works

vidarh 1y ago
Unless your JIT can analyse the full code, a transition between byte code and native code is often costly because the JIT won't be able to optimize the full path. Once your JIT generates good enough code, it then becomes faster to avoid that transition even in cases when in isolation native code might still be faster. EDIT: Note that this isn't an inherent limit. You could write a JIT that could analyze the compiled C code too. It's just that it's much harder to do.
- ksec 1y ago
  And that is what TruffleRuby did. I had wished there is a subset of Ruby that could be compiled to C. And then all gems should be written in that instead. I remember a few people tried but failed though. Have to dig up the old HN threads again.
  - vidarh 1y ago
    Compiling a subset of Ruby to C wouldn't be that hard, but making it compile to C that is fast enough to be worth it is. Not because the Ruby VM is particularly fast, but because the "naive" way of compiling Ruby to C still incurs almost all of the overhead. E.g. TruffleRuby is fast in part because it will do things like try to avoid method calls for built in types where the standard operations haven't been overridden, but that requires a lot of extra machinery... So I'm not sure how much compiling to C would help for gems that use C to speed things up. I think maybe an easier target would be to compile C to a slightly augmented Ruby bytecode. If you control the C compiler you could do things like make C code follow the Ruby calling convention until/unless calling external C code, and avoid a lot of stack overhead.
  - pjmlp 1y ago
    Not everyone failed, see RubyMotion. However they decided it was more useful as a commercial product.
molf 1y ago
C itself is fast; it's calls to C from Ruby that are slow. [1] Crossing the Ruby -> C boundary means that a JIT compiler cannot optimize the code as much; because it cannot alter or inline the C code methods. Counterintuitively this means that rewriting (certain?) built-in methods in Ruby leads to performance gains when using YJIT. [2] [1]: https://railsatscale.com/2023-08-29-ruby-outperforms-c/ https://railsatscale.com/2023-08-29-ruby-outperforms-c/ [2]: https://jpcamara.com/2024/12/01/speeding-up-ruby.html https://jpcamara.com/2024/12/01/speeding-up-ruby.html
nightpool 1y ago
The sibling comments mention that C is used in a lot of places in Ruby that incur cross-language overheads, which is true, but it's also just true that in general, even ignoring this overhead, JIT'd functions are going to be faster then their comparable C functions, because 1) they have more profiling information to be able to work from, 2) they have more type information, and (as a consequence of 1&2) 3) they're more likely to be monomorphized, and the compiler is more able to inline specialized variants of them into different chunks of the code. Among other optimizations!
- uticus 1y ago
  > ...they have more profiling information to be able to work from... more type information... more likely to be monomorphized, and the compiler is more able to inline specialized variants of them into different chunks of the code. this is fascinating to me. i always assumed C had everything in the language that was needed for the compiler to use. in other words, the compiler may have a lot to work through, but the pieces are all available. but this makes it sound like JIT'd functions provide more info to the compiler (more pieces to work with). is there another language besides C that does have language features to indicate to the compiler how to make things as performant as possible?
  - foobazgt 1y ago
    JITs have runtime information that static compilers do not. Sometimes that's not a huge benefit, but it can often have massive performance implications. For example, a JIT can inline dynamically loaded code into your own code. That sounds unusual, but it's actually ultra-common in practice. For example, this shows up in something as mundane and simple as configurable logging.
  - dhruvrajvanshi 1y ago
    A very simple way to think about is that if an intrinsic is written in C, the JIT can't easily inline it, whereas all ruby code can be inlined. Inlining is the most important optimization that enables other optimizations. It's not necessarily the fact that C doesn't have enough information, it's just that the JIT can reason about Ruby code better than it can about C code. To the JIT, C code is just some function which does things and the only thing it can do with it is to call it. On the other hand, a Ruby function's bytecode is available to the jit, so if it sees fit, it can copy paste the function body into the call site and eliminiate the function call overhead. Further, after the inlining, it can apply a lot of further optimizations across what was previously a function boundary. In theory, you could have a way to "compile" the C intrinsics into the JIT's IR directly and that would also give you similar results.

by uticus 1y ago

vidarh 1y ago

Unless your JIT can analyse the full code, a transition between byte code and native code is often costly because the JIT won't be able to optimize the full path. Once your JIT generates good enough code, it then becomes faster to avoid that transition even in cases when in isolation native code might still be faster. EDIT: Note that this isn't an inherent limit. You could write a JIT that could analyze the compiled C code too. It's just that it's much harder to do.

ksec 1y ago
And that is what TruffleRuby did. I had wished there is a subset of Ruby that could be compiled to C. And then all gems should be written in that instead. I remember a few people tried but failed though. Have to dig up the old HN threads again.
- vidarh 1y ago
  Compiling a subset of Ruby to C wouldn't be that hard, but making it compile to C that is fast enough to be worth it is. Not because the Ruby VM is particularly fast, but because the "naive" way of compiling Ruby to C still incurs almost all of the overhead. E.g. TruffleRuby is fast in part because it will do things like try to avoid method calls for built in types where the standard operations haven't been overridden, but that requires a lot of extra machinery... So I'm not sure how much compiling to C would help for gems that use C to speed things up. I think maybe an easier target would be to compile C to a slightly augmented Ruby bytecode. If you control the C compiler you could do things like make C code follow the Ruby calling convention until/unless calling external C code, and avoid a lot of stack overhead.
- pjmlp 1y ago
  Not everyone failed, see RubyMotion. However they decided it was more useful as a commercial product.

molf 1y ago

C itself is fast; it's calls to C from Ruby that are slow. [1] Crossing the Ruby -> C boundary means that a JIT compiler cannot optimize the code as much; because it cannot alter or inline the C code methods. Counterintuitively this means that rewriting (certain?) built-in methods in Ruby leads to performance gains when using YJIT. [2] [1]: https://railsatscale.com/2023-08-29-ruby-outperforms-c/ https://railsatscale.com/2023-08-29-ruby-outperforms-c/ [2]: https://jpcamara.com/2024/12/01/speeding-up-ruby.html https://jpcamara.com/2024/12/01/speeding-up-ruby.html

nightpool 1y ago

The sibling comments mention that C is used in a lot of places in Ruby that incur cross-language overheads, which is true, but it's also just true that in general, even ignoring this overhead, JIT'd functions are going to be faster then their comparable C functions, because 1) they have more profiling information to be able to work from, 2) they have more type information, and (as a consequence of 1&2) 3) they're more likely to be monomorphized, and the compiler is more able to inline specialized variants of them into different chunks of the code. Among other optimizations!

uticus 1y ago
> ...they have more profiling information to be able to work from... more type information... more likely to be monomorphized, and the compiler is more able to inline specialized variants of them into different chunks of the code. this is fascinating to me. i always assumed C had everything in the language that was needed for the compiler to use. in other words, the compiler may have a lot to work through, but the pieces are all available. but this makes it sound like JIT'd functions provide more info to the compiler (more pieces to work with). is there another language besides C that does have language features to indicate to the compiler how to make things as performant as possible?
- foobazgt 1y ago
  JITs have runtime information that static compilers do not. Sometimes that's not a huge benefit, but it can often have massive performance implications. For example, a JIT can inline dynamically loaded code into your own code. That sounds unusual, but it's actually ultra-common in practice. For example, this shows up in something as mundane and simple as configurable logging.
- dhruvrajvanshi 1y ago
  A very simple way to think about is that if an intrinsic is written in C, the JIT can't easily inline it, whereas all ruby code can be inlined. Inlining is the most important optimization that enables other optimizations. It's not necessarily the fact that C doesn't have enough information, it's just that the JIT can reason about Ruby code better than it can about C code. To the JIT, C code is just some function which does things and the only thing it can do with it is to call it. On the other hand, a Ruby function's bytecode is available to the jit, so if it sees fit, it can copy paste the function body into the call site and eliminiate the function call overhead. Further, after the inlining, it can apply a lot of further optimizations across what was previously a function boundary. In theory, you could have a way to "compile" the C intrinsics into the JIT's IR directly and that would also give you similar results.