3 ms·

> It's already the standard way in Ruby, even in Rails. Jeremy Evans did the template optimization you're describing using frozen strings 3+ years ago in Erubi

> It's already the standard way in Ruby, even in Rails. Jeremy Evans did the template optimization you're describing using frozen strings 3+ years ago in Erubi and it was adopted by Rails just a few months later. Yes, I was aware of the frozen strings optimization. Thanks for confirming! The part I am not fully aware though is in regards to the output buffer, which is a separate optimization. Is the output of a template being stored in an array that is writev-ed to the socket? Or is a template still rendered to a large string? Or maybe something in the middle? If you don't mind me asking, what is OOBC? My Google-fu failed me. In any case, 0.5% is quite good, although that is also dependent on how much garbage the app generates. And as you know some libs/frameworks tend to be more mindful of that than others. :) > Are you talking about generating better bytecode based on the semantics of Elixir or a bytecode optimization pass? Do you mean the VM optimizations I was working on? I was mostly AOT compiling the bytecode of a function. That removes the VM dispatch overhead which further helps with CPU speculation. I see very good results on recursive functions but not much beyond that.

jashmatthews 7y ago
It allocates a single "buffer" string. IO.write on Ruby does support using writev but the fairly performance difference hasn't motivated anyone to use it for template rendering yet. Ahhh sorry I meant Out Of Band GC. > Do you mean the VM optimizations I was working on? I was mostly AOT compiling the bytecode of a function. That removes the VM dispatch overhead which further helps with CPU speculation. I see very good results on recursive functions but not much beyond that. It's extremely difficult to get a good return on this approach for two main reasons: instruction dispatch overhead is a very small part of the VM overhead for VMs like BEAM or YARV https://www.sba-research.org/wp-content/uploads/publications/bytecode09.pdf https://www.sba-research.org/wp-content/uploads/publications... and Haswell onwards have no problems branch predicting the dispatch loop https://hal.inria.fr/hal-01100647/document https://hal.inria.fr/hal-01100647/document Sadly this is also why the CRuby 2.6 JIT isn't very effective either. V8 removed their baseline JIT because it wasn't worth it. Firefox is considering doing the same. JSC has a more complex but effective baseline JIT. Implementing one for BEAM would be quite a lot of work compared to a minor performance improvement.
- josevalim 7y ago
  Using a string buffer wouldn't mean that the different dynamic and static parts of the template still have to be copied? Imagine this template: foo <%= expr_that_returns_hello() %> bar Imagine it builds its content like this: output_buffer = "" output_buffer << "foo" output_buffer << expr_that_returns_hello() output_buffer << "bar" With frozen strings, it means "foo" and "bar" won't have to be instantiated on every render, but they are still copied when written to the buffer (unless the buffer uses memory references to frozen strings, akin to a rope). Now imagine the output_buffer is an array, it means less copying, but support needs to be added upstream and other places to handle them. You would also need writev support for arrays (or at least convert them to an iovec or similar). Does this make sense or am I missing something obvious? Btw, sorry for all of the questions and thanks for the linked papers. Now I have something to read on the weekend. :)
  - jashmatthews 7y ago
    TruffleRuby uses ropes to do exactly this. It can even optimize away the buffer string instance: https://chrisseaton.com/truffleruby/ropes-manlang.pdf https://chrisseaton.com/truffleruby/ropes-manlang.pdf CRuby makes a bunch of memcpy calls but compared to everything else, they're fast enough they don't seem to matter much. I've run perf on CRuby running a bunch of different web app type tasks (even rendering huge JSON payloads) and it's never shown up. You can see an example here: https://github.com/k0kubun/railsbench#perf https://github.com/k0kubun/railsbench#perf Perf output for when running Roda looks similar but vm_exec_core and gc_sweep_step are even more of a bottleneck.
    - josevalim 7y ago
      To be honest, I think you are underplaying the value of the sharing optimization (be it via ropes, array buffers, etc) given it can considerably reduce the memory usage (which may put less pressure on the GC). This is because sharing reduces the memory usage in more than half. Without sharing (but with frozen strings), the memory allocation is `1 * static_size + 2 * dynamic_size`. I.e. the cost is each static part, which is copied, and twice the dynamic size, which is created and then copied (I am assuming that most of the dynamic content is allocated on the request life-cycle). With sharing, the cost becomes `0 * static_size + 1 * dynamic_size`. Eliminating the static allocation is particularly important in loops. If you have something like: <% for ... %> pre <%= dynamic %> post <% end %> The factor of `n` applies to both static and dynamic, so you have `n * static_size + 2 * n * dynamic_size` when not sharing but only `n * dynamic_size` when sharing. So you can completely remove a linear component from the equation. You could say that if you use an array or rope, there is still an allocation as the rope or array grows, but that is generally much smaller than the strings being concatenated. It can be preallocated too if you compute the amount of static and dynamic parts when the template is compiled. So while memcpy is fast enough that it wouldn't show during profiling (except maybe by increased GC times), you should see measurable benefits in terms of memory usage. Unfortunately I cannot prove this in practice in the context of Ruby, so I will gladly accept if you think I am overplaying the value of the optimization. :) EDIT: taking a glance at the TruffleRuby rope paper, it seems templating with erb is around 3x faster once they moved to ropes.

jashmatthews 7y ago

It allocates a single "buffer" string. IO.write on Ruby does support using writev but the fairly performance difference hasn't motivated anyone to use it for template rendering yet. Ahhh sorry I meant Out Of Band GC. > Do you mean the VM optimizations I was working on? I was mostly AOT compiling the bytecode of a function. That removes the VM dispatch overhead which further helps with CPU speculation. I see very good results on recursive functions but not much beyond that. It's extremely difficult to get a good return on this approach for two main reasons: instruction dispatch overhead is a very small part of the VM overhead for VMs like BEAM or YARV https://www.sba-research.org/wp-content/uploads/publications/bytecode09.pdf https://www.sba-research.org/wp-content/uploads/publications... and Haswell onwards have no problems branch predicting the dispatch loop https://hal.inria.fr/hal-01100647/document https://hal.inria.fr/hal-01100647/document Sadly this is also why the CRuby 2.6 JIT isn't very effective either. V8 removed their baseline JIT because it wasn't worth it. Firefox is considering doing the same. JSC has a more complex but effective baseline JIT. Implementing one for BEAM would be quite a lot of work compared to a minor performance improvement.

josevalim 7y ago
Using a string buffer wouldn't mean that the different dynamic and static parts of the template still have to be copied? Imagine this template: foo <%= expr_that_returns_hello() %> bar Imagine it builds its content like this: output_buffer = "" output_buffer << "foo" output_buffer << expr_that_returns_hello() output_buffer << "bar" With frozen strings, it means "foo" and "bar" won't have to be instantiated on every render, but they are still copied when written to the buffer (unless the buffer uses memory references to frozen strings, akin to a rope). Now imagine the output_buffer is an array, it means less copying, but support needs to be added upstream and other places to handle them. You would also need writev support for arrays (or at least convert them to an iovec or similar). Does this make sense or am I missing something obvious? Btw, sorry for all of the questions and thanks for the linked papers. Now I have something to read on the weekend. :)
- jashmatthews 7y ago
  TruffleRuby uses ropes to do exactly this. It can even optimize away the buffer string instance: https://chrisseaton.com/truffleruby/ropes-manlang.pdf https://chrisseaton.com/truffleruby/ropes-manlang.pdf CRuby makes a bunch of memcpy calls but compared to everything else, they're fast enough they don't seem to matter much. I've run perf on CRuby running a bunch of different web app type tasks (even rendering huge JSON payloads) and it's never shown up. You can see an example here: https://github.com/k0kubun/railsbench#perf https://github.com/k0kubun/railsbench#perf Perf output for when running Roda looks similar but vm_exec_core and gc_sweep_step are even more of a bottleneck.
  - josevalim 7y ago
    To be honest, I think you are underplaying the value of the sharing optimization (be it via ropes, array buffers, etc) given it can considerably reduce the memory usage (which may put less pressure on the GC). This is because sharing reduces the memory usage in more than half. Without sharing (but with frozen strings), the memory allocation is `1 * static_size + 2 * dynamic_size`. I.e. the cost is each static part, which is copied, and twice the dynamic size, which is created and then copied (I am assuming that most of the dynamic content is allocated on the request life-cycle). With sharing, the cost becomes `0 * static_size + 1 * dynamic_size`. Eliminating the static allocation is particularly important in loops. If you have something like: <% for ... %> pre <%= dynamic %> post <% end %> The factor of `n` applies to both static and dynamic, so you have `n * static_size + 2 * n * dynamic_size` when not sharing but only `n * dynamic_size` when sharing. So you can completely remove a linear component from the equation. You could say that if you use an array or rope, there is still an allocation as the rope or array grows, but that is generally much smaller than the strings being concatenated. It can be preallocated too if you compute the amount of static and dynamic parts when the template is compiled. So while memcpy is fast enough that it wouldn't show during profiling (except maybe by increased GC times), you should see measurable benefits in terms of memory usage. Unfortunately I cannot prove this in practice in the context of Ruby, so I will gladly accept if you think I am overplaying the value of the optimization. :) EDIT: taking a glance at the TruffleRuby rope paper, it seems templating with erb is around 3x faster once they moved to ropes.