I've actually found that a lot of I/O bottlenecks are often Ruby related, for the fact that Ruby doesn't give you a lot of great options in terms of doing that

I've actually found that a lot of I/O bottlenecks are often Ruby related, for the fact that Ruby doesn't give you a lot of great options in terms of doing that I/O in a non-blocking way. If your Ruby program has to do things like fetch data from upstream via a network request, you have to be very careful not to cause request saturation / queueing if that upstream request starts timing out. Even a well-behaving upstream can cause problems if your user patterns require you to hit it a lot. Shopify was fortunate that we had a lot of spare money flying around and could just scale the shit out of things horizontally, but that's a crude solution. Granted, there's been a lot of interesting work done on Ruby concurrency lately, but it's far from a solved problem at the language level. (background: former Shopify EM)