4 ms·

> I think it's time to stop repeating that it's likely that a Rails app is I/O-bound rather than CPU-bound, my experience leads me to think the reverse is curre

> I think it's time to stop repeating that it's likely that a Rails app is I/O-bound rather than CPU-bound, my experience leads me to think the reverse is currently likely. I'd like to offer an anecdotal counterpoint. I worked at Shopify for several years. The big problems were always due to I/O. Usually that was in the form of long-running network calls. You're calling the FedEx API for shipping rates in the cart. All of a sudden FedEx starts taking 5 seconds instead of 1 second, and all of your worker capacity evaporates. That kind of thing. Or, if databases are more your speed, we faced another problem with the GraphQL API needing to rattle off a bunch of requests to the DB, without being able to do so in parallel. So the more different slices of data you needed to fetch, the longer the single GraphQL call would take. We spent significant amounts of time finding ways to work around Rail's inability to do these sorts of things in parallel (or at least, the inability to do it without lots and lots of Spell-o-Tape). Writing proxy services in other languages that could parallelize an incoming request and fan it out to the actual Ruby boxes, etc. I like Ruby but I feel like it's pretty unequipped to deal with the problem of long-running I/O and worker saturation. There's work happening there but it will be a long time before you have the same ease-of-use as languages with better concurrency support out of the box. You could argue that these sorts of long-running calls are not the norm, but in my experience it's way more common for a web app to need to perform some call to an external service at some point during its lifetime than not, and that's not a use case that Ruby/Rails are built to elegantly handle.

deleted 4y ago
[deleted]
neon_electro 4y ago
I'm curious if you could share more on the GraphQL part of the situation you describe here, and if being architected on your more traditional "RESTful" (however you want to define that, but usually Rails defaults get you a wonderful implementation of "RESTful") API convention would have made any difference in the performance issues you encountered at Shopify.
- notpachet 4y ago
  We didn't have the same problem with the old REST API, because the requests were naturally sliced by whatever resource type people were requesting. Whereas the GraphQL API allowed clients to request big slices of disparate data all in the same request. If we had (for whatever reason) exposed a REST API that fetched lots of different slices of data and combined them like GraphQL does, those requests would have faced the same problem. So it was really due to the nature of GraphQL that the problem materialized, rather than being a GraphQL vs REST issue per se.
jrochkind1 4y ago
Thanks for the anecdotal counterpoint, that is what I was looking for! I think we're talking about different things though. I think OP was suggesting/questioning whether a typical Rails apps may spend 90% of it's overall time in I/O. I am suggesting the reverse, that typical Rails apps (so long as they don't have bugs where they fetch more than they need) will spend spend 60-80% of their time in CPU rather than I/O. I am curious if this is true of shoppify's app; my guess is it still would be (especially cause I think shopify will not have very many bugs where it's fetching things inefficiently or that it doesn't need to fetch, shopify will be higher-quality than a random rails app). You are suggesting, I think, that at shopify atypical, problem requests were always due to I/O. I think all of this can be simultaneously true. I think a side issue to what I'm talking about, but: > We spent significant amounts of time finding ways to work around Rail's inability to do these sorts of things in parallel Rails 7 actually recently introduced a feature to do multiple ActiveRecord fetches concurrently... perhaps motivated by/developed by Shopify? https://pawelurbanek.com/rails-load-async https://pawelurbanek.com/rails-load-async I haven't used that one yet. In the past when I've needed to do things like external API requests in parallel, I have had success just using futures/promises from Ruby::Concurrent, without even telling Rails about it. I've had to do this only rarely, but I haven't run into too much trouble. But one thing you can run into if your threads try to store things in ActiveRecord themselves, is running out of database connections with Rails "long connection checkout" model, involving explicit handling of pool checkouts once you do your own threads. Sequel (vs ActiveRecord) has a much more connection-efficient way of pooling database connections, where they are transparently checked out of the pool, under the hood without you usually needing to manage it explicitly, for only as long as it takes to execute a single statement. When doing heavily concurrent things with AR, I have often wished it used Sequel's concurrency/connection-pooling model.
- joshmn 4y ago
  So many familiar faces in this thread. :) With Ruby::Concurrent, I found similar weird-isms until I wrapped in `Rails.application.executor`. Rails has its quirks and purists can be put off by all the "extra work" that needs to happen for things to play nicely together. To that I say: Rails gives me superpowers and I need to abuse them appropriately.
byroot 4y ago
> Usually that was in the form of long-running network calls. That's why doing a network call to another service in the middle of a request is pretty much banned from the monolith. Anything that doesn't have a strict SLO is done from background jobs that can take longer, be retried etc. Now you mention FedEx so I presume you were working on the shipping service, which by essence is kind of an API bridge so it's probably why it was deemed acceptable there, but that's far cry from what a typical Rails app look like, except maybe in companies that do a lot of micro-services and are forced to do inline requests. > that's not a use case that Ruby/Rails are built to elegantly handle. I'd argue the contrary, the elegant way to handle this is to perform these calls from background jobs. Look at shipit [0] for instance, it's syncing with the GitHub API constantly but doesn't do a single API call from inside a request cycle, every single API call is handled asynchronously from background jobs. [0] https://github.com/Shopify/shipit-engine/ https://github.com/Shopify/shipit-engine/
- notpachet 4y ago
  I was actually referring to background job workers in my comment. Queuing is still queuing, whether its request queuing due to a shortage of web workers or job queuing due to a shortage of job workers. Yes, there are more levers one can pull for job workers, and it's probably easier to horizontally scale those workers than web workers for various reasons. But regardless of which workers are performing the long-running I/O, there's still a hard bottleneck imposed by the number of available workers. They're still going to inefficiently block while waiting for the I/O to complete. The bottleneck hasn't been truly eliminated; it's just been punted somewhere else in the application architecture where it can be better mitigated. Background jobs may be the most elegant solution for handling long-running I/O in a typical Rails app, but that's still less elegant than simply performing those requests inline and not having to worry about all the additional moving parts that the jobs entail.
  - byroot 4y ago
    > that's still less elegant than simply performing those requests inline and not having to worry about all the additional moving parts that the jobs entail. I strongly disagree here. Going through a persisted queue gives you lot of tools to manage that queuing. If you were to just spawn a goroutine or some similar async construct you lose persistence, lots of control on retries, resiliency by isolation etc. When you have "in process jobs" re-deploying the service become a nightmare as it becomes extremely muddy how long a request can legitimately take. Whereas if you properly defer these slow IOs to a queue, and only allow fast transactional request, you can then have a very strict request timeout which is really key for reliability of the service.

deleted 4y ago

[deleted]

neon_electro 4y ago

I'm curious if you could share more on the GraphQL part of the situation you describe here, and if being architected on your more traditional "RESTful" (however you want to define that, but usually Rails defaults get you a wonderful implementation of "RESTful") API convention would have made any difference in the performance issues you encountered at Shopify.

notpachet 4y ago
We didn't have the same problem with the old REST API, because the requests were naturally sliced by whatever resource type people were requesting. Whereas the GraphQL API allowed clients to request big slices of disparate data all in the same request. If we had (for whatever reason) exposed a REST API that fetched lots of different slices of data and combined them like GraphQL does, those requests would have faced the same problem. So it was really due to the nature of GraphQL that the problem materialized, rather than being a GraphQL vs REST issue per se.

jrochkind1 4y ago

Thanks for the anecdotal counterpoint, that is what I was looking for! I think we're talking about different things though. I think OP was suggesting/questioning whether a typical Rails apps may spend 90% of it's overall time in I/O. I am suggesting the reverse, that typical Rails apps (so long as they don't have bugs where they fetch more than they need) will spend spend 60-80% of their time in CPU rather than I/O. I am curious if this is true of shoppify's app; my guess is it still would be (especially cause I think shopify will not have very many bugs where it's fetching things inefficiently or that it doesn't need to fetch, shopify will be higher-quality than a random rails app). You are suggesting, I think, that at shopify atypical, problem requests were always due to I/O. I think all of this can be simultaneously true. I think a side issue to what I'm talking about, but: > We spent significant amounts of time finding ways to work around Rail's inability to do these sorts of things in parallel Rails 7 actually recently introduced a feature to do multiple ActiveRecord fetches concurrently... perhaps motivated by/developed by Shopify? https://pawelurbanek.com/rails-load-async https://pawelurbanek.com/rails-load-async I haven't used that one yet. In the past when I've needed to do things like external API requests in parallel, I have had success just using futures/promises from Ruby::Concurrent, without even telling Rails about it. I've had to do this only rarely, but I haven't run into too much trouble. But one thing you can run into if your threads try to store things in ActiveRecord themselves, is running out of database connections with Rails "long connection checkout" model, involving explicit handling of pool checkouts once you do your own threads. Sequel (vs ActiveRecord) has a much more connection-efficient way of pooling database connections, where they are transparently checked out of the pool, under the hood without you usually needing to manage it explicitly, for only as long as it takes to execute a single statement. When doing heavily concurrent things with AR, I have often wished it used Sequel's concurrency/connection-pooling model.

joshmn 4y ago
So many familiar faces in this thread. :) With Ruby::Concurrent, I found similar weird-isms until I wrapped in `Rails.application.executor`. Rails has its quirks and purists can be put off by all the "extra work" that needs to happen for things to play nicely together. To that I say: Rails gives me superpowers and I need to abuse them appropriately.

byroot 4y ago

> Usually that was in the form of long-running network calls. That's why doing a network call to another service in the middle of a request is pretty much banned from the monolith. Anything that doesn't have a strict SLO is done from background jobs that can take longer, be retried etc. Now you mention FedEx so I presume you were working on the shipping service, which by essence is kind of an API bridge so it's probably why it was deemed acceptable there, but that's far cry from what a typical Rails app look like, except maybe in companies that do a lot of micro-services and are forced to do inline requests. > that's not a use case that Ruby/Rails are built to elegantly handle. I'd argue the contrary, the elegant way to handle this is to perform these calls from background jobs. Look at shipit [0] for instance, it's syncing with the GitHub API constantly but doesn't do a single API call from inside a request cycle, every single API call is handled asynchronously from background jobs. [0] https://github.com/Shopify/shipit-engine/ https://github.com/Shopify/shipit-engine/

notpachet 4y ago
I was actually referring to background job workers in my comment. Queuing is still queuing, whether its request queuing due to a shortage of web workers or job queuing due to a shortage of job workers. Yes, there are more levers one can pull for job workers, and it's probably easier to horizontally scale those workers than web workers for various reasons. But regardless of which workers are performing the long-running I/O, there's still a hard bottleneck imposed by the number of available workers. They're still going to inefficiently block while waiting for the I/O to complete. The bottleneck hasn't been truly eliminated; it's just been punted somewhere else in the application architecture where it can be better mitigated. Background jobs may be the most elegant solution for handling long-running I/O in a typical Rails app, but that's still less elegant than simply performing those requests inline and not having to worry about all the additional moving parts that the jobs entail.
- byroot 4y ago
  > that's still less elegant than simply performing those requests inline and not having to worry about all the additional moving parts that the jobs entail. I strongly disagree here. Going through a persisted queue gives you lot of tools to manage that queuing. If you were to just spawn a goroutine or some similar async construct you lose persistence, lots of control on retries, resiliency by isolation etc. When you have "in process jobs" re-deploying the service become a nightmare as it becomes extremely muddy how long a request can legitimately take. Whereas if you properly defer these slow IOs to a queue, and only allow fast transactional request, you can then have a very strict request timeout which is really key for reliability of the service.