4 ms·

I didn't get the point about YJIT only helping 15-30% in real apps while IO-less benchmarks see 2-3x. It seems like a counterpoint to the main argument of the a

I didn't get the point about YJIT only helping 15-30% in real apps while IO-less benchmarks see 2-3x. It seems like a counterpoint to the main argument of the article. Instead of guessing if the DB is performing badly by looking at Rails, why not go directly to the DB?

byroot 2y ago
Someone else pointed the same thing on bluesky, so I guess this part of my post isn't very clear. First, I/O bound isn't necessarily a very well defined term, but generally speaking, it implies that you can't substantially speedup such system by speeding up code execution. The fact that YJIT did yield substantial performance improvement in the real world suggest many Rails apps aren't in fact strictly I/O bound. Now about the 15-30% vs 2-3x, what I mean by that is the benchmark where YJIT yield this much are mostly micro-benchmarks, on more complex and heterogenous code, the gains are much smaller, hence we can assume the "Ruby" part of these applications wasn't improved by this much. So for YJIT to be able to yield `15-30%` latency gains, this latency must have been in large part composed of Ruby execution. One last thing to note is that YJIT can only speedup pure Ruby code, in a Rails application, a large part of the CPU time isn't in pure Ruby code, but in various methods implemented in C that YJIT can't speedup. Ultimately the lobste.rs maintainer kindly offered to run an experiment in production, so we should soon see if my assumptions hold true or if I was way off, at least for that particular app: https://github.com/lobsters/lobsters/pull/1442 https://github.com/lobsters/lobsters/pull/1442 Edit: > Instead of guessing if the DB is performing badly by looking at Rails That isn't really the topic though. The question is more about how much Ruby's GVL is actually released, and its implications.
SkiFire13 2y ago
The point was that 15-30% is a very big speedup for a change that impacts what is supposed to take a very small part of the total time. See also Amdahl's law [1] Suppose that the CPU-intensive part of the program was actually speed up by 3x. This means that previously it took 1.5x as much time as the one that was saved, so thoughly on the order of 22-45% of the total time of your program. In practice the speedup was probably lower than 3x, which means the 22-45% figure would be even higher. These are really really high number for a system that was supposed to spend most of the time (>90%) blocked in IO (which was not speed up by YJIT). For comparison, if the CPU-intensive part actually took 10% of the time, then 10% would be maximum speedup you could get (which is way lower than 15-30%) and a 3x speedup would only result in a ~6% speedup. [1]: https://en.m.wikipedia.org/wiki/Amdahl%27s_law https://en.m.wikipedia.org/wiki/Amdahl%27s_law
ksec 2y ago
If your Application is 50/50 in CPU / IO at a 100ms request. Getting 30% Reduction in total response time would mean the request is now only 70ms, DB stays the same at 50ms, your CPU is now only 20ms. That is 2.5x speed up compare to the 50ms previously. Exactly inline with the 2-3x IO-less benchmarks.

byroot 2y ago

Someone else pointed the same thing on bluesky, so I guess this part of my post isn't very clear. First, I/O bound isn't necessarily a very well defined term, but generally speaking, it implies that you can't substantially speedup such system by speeding up code execution. The fact that YJIT did yield substantial performance improvement in the real world suggest many Rails apps aren't in fact strictly I/O bound. Now about the 15-30% vs 2-3x, what I mean by that is the benchmark where YJIT yield this much are mostly micro-benchmarks, on more complex and heterogenous code, the gains are much smaller, hence we can assume the "Ruby" part of these applications wasn't improved by this much. So for YJIT to be able to yield `15-30%` latency gains, this latency must have been in large part composed of Ruby execution. One last thing to note is that YJIT can only speedup pure Ruby code, in a Rails application, a large part of the CPU time isn't in pure Ruby code, but in various methods implemented in C that YJIT can't speedup. Ultimately the lobste.rs maintainer kindly offered to run an experiment in production, so we should soon see if my assumptions hold true or if I was way off, at least for that particular app: https://github.com/lobsters/lobsters/pull/1442 https://github.com/lobsters/lobsters/pull/1442 Edit: > Instead of guessing if the DB is performing badly by looking at Rails That isn't really the topic though. The question is more about how much Ruby's GVL is actually released, and its implications.

SkiFire13 2y ago

The point was that 15-30% is a very big speedup for a change that impacts what is supposed to take a very small part of the total time. See also Amdahl's law [1] Suppose that the CPU-intensive part of the program was actually speed up by 3x. This means that previously it took 1.5x as much time as the one that was saved, so thoughly on the order of 22-45% of the total time of your program. In practice the speedup was probably lower than 3x, which means the 22-45% figure would be even higher. These are really really high number for a system that was supposed to spend most of the time (>90%) blocked in IO (which was not speed up by YJIT). For comparison, if the CPU-intensive part actually took 10% of the time, then 10% would be maximum speedup you could get (which is way lower than 15-30%) and a 3x speedup would only result in a ~6% speedup. [1]: https://en.m.wikipedia.org/wiki/Amdahl%27s_law https://en.m.wikipedia.org/wiki/Amdahl%27s_law

ksec 2y ago

If your Application is 50/50 in CPU / IO at a 100ms request. Getting 30% Reduction in total response time would mean the request is now only 70ms, DB stays the same at 50ms, your CPU is now only 20ms. That is 2.5x speed up compare to the 50ms previously. Exactly inline with the 2-3x IO-less benchmarks.