3 ms·
Is there a name for this syndrome? The same thing happened at Github. Ruby and Rails are slow enough in production that they dedicate a specialized, long-runnin
by stevebmark 4y ago
Is there a name for this syndrome? The same thing happened at Github. Ruby and Rails are slow enough in production that they dedicate a specialized, long-running internal team to it. This team doesn't contribute directly to the product. They hire core Ruby and Rails maintainers, also not to contribute to the product, just to try to help improve their slow software. It's a dream job that shouldn't exist.
If you tell me that Ruby and Rails scaled for Github and Shopify, be careful using that as an argument for choosing Rails for your company. Rails definitely didn't scale for Github, and doesn't appear like it scaled for Shopify, until a manager greenlit a year long JIT project unrelated to Shopify. If your company has as many resources, free cash, klout to hire core Ruby maintainers, and enough downtime you can throw a few engineers at a non-product problem for a few years, then sure, go ahead and use that reasoning.
- nirvdrum 4y agoDon't most large products have teams working on performance? With a VM-based language like Ruby, speeding up the VM is a sound strategy because it works across all of your applications. There are teams embedded in corporations working on the JVM, the Python VM, the PHP VM, and so on (sometimes brand new VM implementations). There are teams working on improving performance of the standard library and compiler in C, C++, Rust, and Go. There are teams working improving performance of applications in all of those languages. Sure, if you're a startup and you can't afford it, you probably shouldn't be spending time speeding up your language. But, you're also unlikely to experience performance issues under your lighter load that can't be attributed to your application.
- xutopia 4y agoShopify like many other successful Ruby shops were successful because of the speed of development. The cost of hosting is secondary and any company the size of Shopify can afford to involve a small team to reduce hosting costs... with impact for the developers happiness to boot.
- ezekg 4y agoI wonder if YJIT actually did decrease infrastructure costs for Shopify, seeing as it requires ~3x more memory. For my business, it’s come out about even. But a >30% decrease in response time is worth it, for me at least. I run many small servers (2 processes each), but IIRC, Shopify runs beefy servers. So may be different for them.
- byroot 4y ago> as it requires ~3x more memory. Not sure were you've seen this, but that's absolutely not normal. The YJIT memory overhead is certainly sizeable, but is generally at least an order of magnitude smaller than that.
- ezekg 4y agoReddit. Per this discussion here: https://old.reddit.com/r/ruby/comments/zzbrch/how_much_memory_is_yjit_supposed_to_consume/j2cxy9d/ https://old.reddit.com/r/ruby/comments/zzbrch/how_much_memor.... My production workload shows similar results. The YJIT app's memory sits at about 2.4x what my non-YJIT app utilizes. Sidekiq sits at 3x.
- bcrosby95 4y agoLanguages don't scale (out), architecture does. Well, maybe Elixir and Erlang scales out, since architecture is baked in. Depending on how performant a language is, it can put off (sometimes forever) your need to scale out, which can simplify things. But both Github and Shopify are beyond the level of traffic that would allow you to get away with only scaling up rather than out.
- djur 4y agoCan you give some examples of companies that "scaled" without dedicating at least some engineering work to improving their runtime?
- stevebmark 4y agoYou should be asking specifically for examples of companies that don't do anything computationally intensive, aren't CPU bound, and still dedicated teams to work on the overall performance of the _language_ itself they use. Not tuning the VM, not tracking down memory leaks, trying to fix the thing itself. Which is the case here.