4 ms·

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-req

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-request arena that is asynchronously freed which would force the programmer not to have any objects that outlive the request, or just pre-allocating memory for a fixed amount of request handling per server instance, or whatever allocation behavior you want to do that is generally possible in C/C++/Zig/Rust/Odin/etc, requires hacking on the language itself. Which means your changes have to go through the Ruby team first. Any additional changes would also need to go through them, which increases the cost of change. Then there is a permanent layer of indirection between your GC callbacks and the semantics of what those callbacks do. Instead of just writing out the custom allocators you want, because that's impossible. How depressing.

jrockway 2y ago
I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)
- bhaak 2y ago
  Startups are not large companies in the beginning. Although I'm not sure what the preferred language for quickly getting a startup up and running would be these days.
  - floating-io 2y ago
    Whatever works and you can find enough developers for. The language (or the rest of the stack even) is rarely a barrier to success. What matters are a good idea, good motivation, and decent availability of competence. JMHO.
- byroot 2y ago
  I don't see how it is imperfect. Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea. Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith. Ultimately, while it's a bit counter intuitive, GCs can perform extremely well in term of throughput. Ruby's GC isn't quite there yet, but still perform quite well and is improving every versions.
  - samatman 2y ago
    > allocating more than the arena can accommodate In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that). It's easy to set them to retain a 'reasonable' allocated capacity when they get reset, for whatever value of reasonable, so big allocation spikes get actually freed, but normal use just moves a pointer back and reuses that memory. I don't see Shopify harvesting a lot of value from a complete Zig rewrite, no. But arenas are basically ideal for the sort of memory use which web servers typically exhibit.
    - simonask 2y ago
      And when the default arena size is often outgrown, you'll known from whatever diagnostics/logging/dashboard solution you are using. Which is incidentally also a great tool when optimizing per-request memory usage. Being explicit about memory has many advantages, and is a strict requirement when scaling.
    - byroot 2y ago
      > something no allocation strategy can fix Well, yes, with a GC when your heap is full, you make space by getting rid of the garbage. Also, with a good GC, allocating is most of the time just bumping a pointer, exactly like an arena, and the collection time is proportional to the number of live objects, which when triggered out of band is basically 0. Hence why I think a well tuned GC really isn't that far off.
- danmur 2y ago
  I don't think using Zig over Python is gonna have the biggest impact in making your next big company successful. It's a drop in the ocean compared to the quality of people you have to actually design and build it.
  - simonask 2y ago
    It doesn't matter for a startup trying to get acquired. It does matter for a company trying to scale its user base while keeping costs down.
  - indulona 2y ago
    That is nonsense. If you can run your code on 10 servers instead of 1k servers, that is an insane time and money saver that could make or break a company.
    - WJW 2y ago
      For most startups the difference between Rust and Ruby is not 10 servers vs 1000 but rather between using 0.1% of a CPU or 1% of a CPU. A single server running Rails will easily scale to hundreds of thousands of daily users. Most companies never get that many users in the first place, and those that do will have the funds to afford rewriting the hottest paths in a more performant language.
    - danmur 2y ago
      I certainly don't claim to be an expert, but I have a hunch that getting to the point where performance becomes a significant factor (in the success or failure of a product) isn't going to be about the choice of language. I also think you're vastly underestimating the performance that good architects can get out of ANY (primary) language through good system design. Good design vs bad design makes the biggest difference in my experience, at least from a technical standpoint. Probably just nonsense though as you say.
xerxes901 2y ago
> Which means your changes have to go through the Ruby team first. Any additional changes would also need to go through them … I do want to pick on this specifically - people can and should be patching open source projects they depend on and deploying them to production (exactly as described in the article). Something being in the language vs in “user” code should be no barrier to improving it.
- simonask 2y ago
  There's a pretty huge difference between implementing a performance optimization that works in your use case, and upstreaming that optimization to be generally usable. The latter is often orders of magnitude more work, and the existing solution is probably chosen to be well suited in general.
- gmueckl 2y ago
  Patching a dependency comes with significant downstream costs. You need to carry the patch forward to new upstream versions . This implies remembering that the dependency was patched, extracting a patch from the existing changed code, and reapplying the patch, fixing comflicts, recompiling the now special version of that dependency and running tests, checking/updating required license notices accordingly. This is in essence another form of technical dept.

jrockway 2y ago

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

bhaak 2y ago
Startups are not large companies in the beginning. Although I'm not sure what the preferred language for quickly getting a startup up and running would be these days.
- floating-io 2y ago
  Whatever works and you can find enough developers for. The language (or the rest of the stack even) is rarely a barrier to success. What matters are a good idea, good motivation, and decent availability of competence. JMHO.
byroot 2y ago
I don't see how it is imperfect. Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea. Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith. Ultimately, while it's a bit counter intuitive, GCs can perform extremely well in term of throughput. Ruby's GC isn't quite there yet, but still perform quite well and is improving every versions.
- samatman 2y ago
  > allocating more than the arena can accommodate In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that). It's easy to set them to retain a 'reasonable' allocated capacity when they get reset, for whatever value of reasonable, so big allocation spikes get actually freed, but normal use just moves a pointer back and reuses that memory. I don't see Shopify harvesting a lot of value from a complete Zig rewrite, no. But arenas are basically ideal for the sort of memory use which web servers typically exhibit.
  - simonask 2y ago
    And when the default arena size is often outgrown, you'll known from whatever diagnostics/logging/dashboard solution you are using. Which is incidentally also a great tool when optimizing per-request memory usage. Being explicit about memory has many advantages, and is a strict requirement when scaling.
  - byroot 2y ago
    > something no allocation strategy can fix Well, yes, with a GC when your heap is full, you make space by getting rid of the garbage. Also, with a good GC, allocating is most of the time just bumping a pointer, exactly like an arena, and the collection time is proportional to the number of live objects, which when triggered out of band is basically 0. Hence why I think a well tuned GC really isn't that far off.
danmur 2y ago
I don't think using Zig over Python is gonna have the biggest impact in making your next big company successful. It's a drop in the ocean compared to the quality of people you have to actually design and build it.
- simonask 2y ago
  It doesn't matter for a startup trying to get acquired. It does matter for a company trying to scale its user base while keeping costs down.
- indulona 2y ago
  That is nonsense. If you can run your code on 10 servers instead of 1k servers, that is an insane time and money saver that could make or break a company.
  - WJW 2y ago
    For most startups the difference between Rust and Ruby is not 10 servers vs 1000 but rather between using 0.1% of a CPU or 1% of a CPU. A single server running Rails will easily scale to hundreds of thousands of daily users. Most companies never get that many users in the first place, and those that do will have the funds to afford rewriting the hottest paths in a more performant language.
  - danmur 2y ago
    I certainly don't claim to be an expert, but I have a hunch that getting to the point where performance becomes a significant factor (in the success or failure of a product) isn't going to be about the choice of language. I also think you're vastly underestimating the performance that good architects can get out of ANY (primary) language through good system design. Good design vs bad design makes the biggest difference in my experience, at least from a technical standpoint. Probably just nonsense though as you say.

xerxes901 2y ago

> Which means your changes have to go through the Ruby team first. Any additional changes would also need to go through them … I do want to pick on this specifically - people can and should be patching open source projects they depend on and deploying them to production (exactly as described in the article). Something being in the language vs in “user” code should be no barrier to improving it.

simonask 2y ago
There's a pretty huge difference between implementing a performance optimization that works in your use case, and upstreaming that optimization to be generally usable. The latter is often orders of magnitude more work, and the existing solution is probably chosen to be well suited in general.
gmueckl 2y ago
Patching a dependency comes with significant downstream costs. You need to carry the patch forward to new upstream versions . This implies remembering that the dependency was patched, extracting a patch from the existing changed code, and reapplying the patch, fixing comflicts, recompiling the now special version of that dependency and running tests, checking/updating required license notices accordingly. This is in essence another form of technical dept.