5 ms·

Also for a practical tip on YJIT memory usage, note that there is a "--yjit-exec-mem-size" option, see https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md

Also for a practical tip on YJIT memory usage, note that there is a "--yjit-exec-mem-size" option, see https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#command-line-options https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#co... for more details. (This command-line argument is mentioned in the paper https://dl.acm.org/doi/10.1145/3617651.3622982 https://dl.acm.org/doi/10.1145/3617651.3622982 but not in this blog post about the paper.) At Heii On-Call https://heiioncall.com/ https://heiioncall.com/ we use: ENV RUBY_YJIT_ENABLE=1 ENV RUBYOPT=--yjit-exec-mem-size=16 in our Dockerfile for our Rails processes.

booleanbetrayal 3y ago
Any recollection on how you arrived at the --yjit-exec-mem-size value? We've been running YJIT in production for some time, but haven't looked into tuning this at all.
- JohnBooty 3y ago
  Not parent poster and do not have production YJIT experience. =) My guess is that you would monitor `RubyVM::YJIT.runtime_stats[:code_region_size]` and/or `RubyVM::YJIT.runtime_stats[:code_gc_count]` so that you can get a feel for a reasonable value for your application, as well as know whether or not the "code GC" is running frequently. https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#performance-tips-for-production-deployments https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#pe...
  - compumike 3y ago
    That’s exactly right. Our code_region_size levels off a bit under 8 MB, so we set the limit to 16. In practice we see code_gc_count stays at 0.
    - booleanbetrayal 3y ago
      Will have to look into this. Thanks for the suggestion!
JohnBooty 3y ago
Wow, that's interesting and it seems a little crazy? From the docs: When JIT code size (RubyVM::YJIT.runtime_stats[:code_region_size]) reaches this value, YJIT triggers "code GC" that frees all JIT code and starts recompiling everything. Compiling code takes some time, so scheduling code GC too frequently slows down your application. Increasing --yjit-exec-mem-size may speed up your application if RubyVM::YJIT.runtime_stats[:code_gc_count] is not 0 or 1. https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#command-line-options https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#co... It just dumps all the JIT-compiled code? I'd expect to see some kind of heuristic or algorithm there... LFU or something. The internals of a JIT are essentially black magic to me, and I know the people working on YJIT are super talented, so I am sure there is a good reason why they just dump everything instead of the least-frequently used stuff. Maybe the overhead of trying frecency outweighs the gains, maybe they just haven't implemented it yet, or maybe it's just a rarely-reached condition. (I hope a YJIT team member sees this, I'm super curious now)
- xerxes901 3y ago
  I don't work on YJIT but I _think_ i know the (or maybe an) answer to this. The code for a JIT'd ruby method isn't contiguous in one location in memory. When a ruby method is first compiled, a straightline path through the method is emitted , and branches are emitted as stub code. When the stub is hit, the incremental compilation of that branch then happens. I believe this is called "lazy basic block versioning". When the stub is hit the code that gets generated is somewhere _else_ in executable memory, not contiguous with the original bit of the method. Because these "lazy basic blocks" are actually quite small, the bookkeeping involved in "where is the code for this ruby method" would actually be an appreciable fraction of the code size itself. Plus you then have to do more bookkeeping to make sure the method you want to GC isn't referred to by the generated code in another method. Since low memory usage is an important YJIT goal, I guess this tradeoff isn't worth it. Maybe someone who knows this better will come along and correct me :)
  - JohnBooty 3y ago
    Seems you hit the nail on the head. Thanks for the informative answer!
ksec 3y ago
Off Topic, but I thought Heii On-Call was done on Crystal? Or did I mixed it with something else?

booleanbetrayal 3y ago

Any recollection on how you arrived at the --yjit-exec-mem-size value? We've been running YJIT in production for some time, but haven't looked into tuning this at all.

JohnBooty 3y ago
Not parent poster and do not have production YJIT experience. =) My guess is that you would monitor `RubyVM::YJIT.runtime_stats[:code_region_size]` and/or `RubyVM::YJIT.runtime_stats[:code_gc_count]` so that you can get a feel for a reasonable value for your application, as well as know whether or not the "code GC" is running frequently. https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#performance-tips-for-production-deployments https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#pe...
- compumike 3y ago
  That’s exactly right. Our code_region_size levels off a bit under 8 MB, so we set the limit to 16. In practice we see code_gc_count stays at 0.
  - booleanbetrayal 3y ago
    Will have to look into this. Thanks for the suggestion!

JohnBooty 3y ago

Wow, that's interesting and it seems a little crazy? From the docs: When JIT code size (RubyVM::YJIT.runtime_stats[:code_region_size]) reaches this value, YJIT triggers "code GC" that frees all JIT code and starts recompiling everything. Compiling code takes some time, so scheduling code GC too frequently slows down your application. Increasing --yjit-exec-mem-size may speed up your application if RubyVM::YJIT.runtime_stats[:code_gc_count] is not 0 or 1. https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#command-line-options https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#co... It just dumps all the JIT-compiled code? I'd expect to see some kind of heuristic or algorithm there... LFU or something. The internals of a JIT are essentially black magic to me, and I know the people working on YJIT are super talented, so I am sure there is a good reason why they just dump everything instead of the least-frequently used stuff. Maybe the overhead of trying frecency outweighs the gains, maybe they just haven't implemented it yet, or maybe it's just a rarely-reached condition. (I hope a YJIT team member sees this, I'm super curious now)

xerxes901 3y ago
I don't work on YJIT but I _think_ i know the (or maybe an) answer to this. The code for a JIT'd ruby method isn't contiguous in one location in memory. When a ruby method is first compiled, a straightline path through the method is emitted , and branches are emitted as stub code. When the stub is hit, the incremental compilation of that branch then happens. I believe this is called "lazy basic block versioning". When the stub is hit the code that gets generated is somewhere _else_ in executable memory, not contiguous with the original bit of the method. Because these "lazy basic blocks" are actually quite small, the bookkeeping involved in "where is the code for this ruby method" would actually be an appreciable fraction of the code size itself. Plus you then have to do more bookkeeping to make sure the method you want to GC isn't referred to by the generated code in another method. Since low memory usage is an important YJIT goal, I guess this tradeoff isn't worth it. Maybe someone who knows this better will come along and correct me :)
- JohnBooty 3y ago
  Seems you hit the nail on the head. Thanks for the informative answer!

ksec 3y ago

Off Topic, but I thought Heii On-Call was done on Crystal? Or did I mixed it with something else?