5 ms·

The Mythical IO-Bound Rails App

227 points by thunderbong 2y ago

pdhborges 2y ago
Very nice discussion. A lot of times I see people assuming that Web Apps are IO bound anyway to justify technical decisions. Nevertheless here am I looking at our stats: 60% is python, 40% is IO and Postgres is getting faster than Python at a faster pace.
gtirloni 2y ago
I didn't get the point about YJIT only helping 15-30% in real apps while IO-less benchmarks see 2-3x. It seems like a counterpoint to the main argument of the article. Instead of guessing if the DB is performing badly by looking at Rails, why not go directly to the DB?
- byroot 2y ago
  Someone else pointed the same thing on bluesky, so I guess this part of my post isn't very clear. First, I/O bound isn't necessarily a very well defined term, but generally speaking, it implies that you can't substantially speedup such system by speeding up code execution. The fact that YJIT did yield substantial performance improvement in the real world suggest many Rails apps aren't in fact strictly I/O bound. Now about the 15-30% vs 2-3x, what I mean by that is the benchmark where YJIT yield this much are mostly micro-benchmarks, on more complex and heterogenous code, the gains are much smaller, hence we can assume the "Ruby" part of these applications wasn't improved by this much. So for YJIT to be able to yield `15-30%` latency gains, this latency must have been in large part composed of Ruby execution. One last thing to note is that YJIT can only speedup pure Ruby code, in a Rails application, a large part of the CPU time isn't in pure Ruby code, but in various methods implemented in C that YJIT can't speedup. Ultimately the lobste.rs maintainer kindly offered to run an experiment in production, so we should soon see if my assumptions hold true or if I was way off, at least for that particular app: https://github.com/lobsters/lobsters/pull/1442 https://github.com/lobsters/lobsters/pull/1442 Edit: > Instead of guessing if the DB is performing badly by looking at Rails That isn't really the topic though. The question is more about how much Ruby's GVL is actually released, and its implications.
- SkiFire13 2y ago
  The point was that 15-30% is a very big speedup for a change that impacts what is supposed to take a very small part of the total time. See also Amdahl's law [1] Suppose that the CPU-intensive part of the program was actually speed up by 3x. This means that previously it took 1.5x as much time as the one that was saved, so thoughly on the order of 22-45% of the total time of your program. In practice the speedup was probably lower than 3x, which means the 22-45% figure would be even higher. These are really really high number for a system that was supposed to spend most of the time (>90%) blocked in IO (which was not speed up by YJIT). For comparison, if the CPU-intensive part actually took 10% of the time, then 10% would be maximum speedup you could get (which is way lower than 15-30%) and a 3x speedup would only result in a ~6% speedup. [1]: https://en.m.wikipedia.org/wiki/Amdahl%27s_law https://en.m.wikipedia.org/wiki/Amdahl%27s_law
- ksec 2y ago
  If your Application is 50/50 in CPU / IO at a 100ms request. Getting 30% Reduction in total response time would mean the request is now only 70ms, DB stays the same at 50ms, your CPU is now only 20ms. That is 2.5x speed up compare to the 50ms previously. Exactly inline with the 2-3x IO-less benchmarks.
rf15 2y ago
Usually it's the inefficiencies of architectural decisions (your algos, layouts, etc.) you put into that rails app.
- scosman 2y ago
  Sometimes. But if you have any real processing to do, it can be hard/impossible to do it quickly in ruby. You end up writing c extensions, or another app server in another language for the hard stuff.
- ndriscoll 2y ago
  Right. Simple test: check the throughput your database can do with bulk queries. Do a single query select of an xml or json struct that outputs the page rails would return (so one rendered page per row). Select enough rows for the query to take 1-10s to get a sense for rendered pages/second. Do the same for inserts/updates. If you're not close to what postgres alone can do with bulk queries, you're obviously not IO bound. Other sanity check: nvme drives are ~10000x faster than hard disks were. Are you doing 10000x more useful disk IO than people were doing with hard disks?
  - anarazel 2y ago
    For typical rails like applications that are "bottlenecked on the database", latency is a much bigger factor than actual query processing performance. Even if careful about placement of the database and "application servers". Just as an example, here's the results of pgbench -S of a single client, with pgbench running on different servers (a single pkey lookup): pgbench on database server: 36233 QPS pgbench on a different system (local 10gbit network): 2860 QPS It's rare to actually have that low latency to the database in the real world IME. If the same pkey lookup is executed utilizing pipelining, I get ~145k QPS from both locally and remotely.
    - ndriscoll 2y ago
      That's still an architectural thing. If CPU isn't an issue, why would you be running your application server on a different machine from your database? Connecting to a unix domain socket will give better latency and throughput and security as long as you can scale vertically. Even if you have it on another machine, it's on the same switch, right (also an architectural choice)? So latency should be sub millisecond, and irrelevant for a single request. So we assume we're talking about many requests, but then why not combine/batch your queries (also an architectural choice)? Now latency (and other things like database latches and fsyncs. Very important.) is amortized and again irrelevant. You should be able to hit close to the pure pg bulk query case (and e.g. a rust application server can do this in practice for me with a 4 core machine and an old SATA SSD with ~50k read IOPS). Point is, the optimal case for building a web page is probably to do pg queries that return the xml of the page (letting you do subselects to combine everything into one query). So see how fast that is, and if your application can't match that, the database is not your bottleneck.
scosman 2y ago
“CPU starvation looks like IO to most eyes” Yes! And add in that you are holding a database connection for longer than needed, putting scaling pressure on your DB, which is the harder thing to scale.
- PaulHoule 2y ago
  I'm not sure if it is the same phenomenon but I was really disappointed with the performance of a python aiohttp server [1] when I went from the application serving an html page with maybe 2-3 small images that would stick in the cache (icons) and instead was serving 20-50 photographic images per page, particularly over the wrong direction of an ADSL connection. I thought this was low CPU but it was not low CPU enough, in the end I switched to gunicorn (running in WSL) with IIS fronting and serving the images (woulda been nginix in a pure-Linux environment) I think the tiny amount of CPU to serve the images was still getting blocked by requests that used a little more CPU. With gunicorn I'm not afraid to add tasks that crunch on the CPU a little harder too. [1] built a lot of mini-apps that worked great before
  - CoolCold 2y ago
    Everything is better with Nginx! As sysadmin I've seen many cases when not letting backend to serve static files drops latency and load significantly. In the worst case, X-accel-redirect is still better than serving through most of the frameworks.
- WJW 2y ago
  Having the connection for longer than needed only puts more pressure on databases on which connections are a scarce resource (basically only postgres, and even then it's easily fixed with pgbouncer). When using a database with a better connection model, holding a connection too long will actually decrease total load on the database because each connection is idle for a larger percentage of the time.
  - CoolCold 2y ago
    Isn't pgbouncer putting extra limitations like prepared statemens use ? Genuine question
Azerty9999 2y ago
There is a highly relevant follow-up post by the author here, too: https://byroot.github.io/ruby/performance/2025/01/23/io-instrumentation.html https://byroot.github.io/ruby/performance/2025/01/23/io-inst...
jupp0r 2y ago
When people say these apps are IO bound, what they actually mean is that they are memory bound. While Rails is waiting for IO due to its lack of concurrency support and the ubiquitous use of global variables in the ecosystem, a passenger worker is taking up >300mb of memory for a medium sized Rails app. Server memory limits the amount of workers you can have waiting like that, which in turn limits overall throughput per server.
- caseyohara 2y ago
  That you mention Passenger makes me wonder how outdated your comment is. The most recent Rails community survey reported only 13% using Passenger. https://railsdeveloper.com/survey/2024/ https://railsdeveloper.com/survey/2024/
- byroot 2y ago
  That may be true for very memory constrained platforms like Heroku, but is a non issue at larger scale. Taking your figure of 300MiB per process, and assuming you might want to run 1.5 process per core to account for IO-wait, that's 450MiB per core (ignoring copy-on-write). If you look at various hosting offering, most offer something like 4GiB of RAM per core (e.g. EC2 "general purpose" instances). As for the lack of concurrency support, Active Record has asynchronous queries since a few versions, and is now even async/fiber compatible. So if you truly have an app that would benefit from more concurrency, you can do it.
- pydry 2y ago
  No, it just means that they spend most of their time waiting around for database and network queries to finish.
- marcosdumay 2y ago
  They mean the app takes data from one place, does a trivial computation, and pushed it at another place. So pushing data around is all the application does. And they don't know both such places are actually faster than the trivial computation because their code runs way slower than the ones at the origin and destination.
karmakaze 2y ago
The discussion in the referenced Rails issue "Set a new default for the Puma thread count"[0] to me is much more telling of IO vs CPU than this simplified post. There are many assumptions here especially not considering anything about the database itself. The Rails issue also considers benchmarks which when benchmarking Rails would configure the database to not be the bottleneck. That's not true of real systems. The advice to use async queries for an IO-bound app could backfire if the reason the queries are slow is because the database is overloaded--adding concurrent queries only increases its thrashing and latency. The best thing to do is consider the whole system. Don't throw the everything that's not the Rails app as IO. Is it actually doing network IO, or is it CPU, memory, or IO bound in the database? Maybe its not even a lack of CPU proper on the database but that it's being wasted on write contention/locks. Only then will you be able to choose the right course. Another way to go is blindly try different configurations and use what works well without full understanding, which is fine until you have an outage you can't explain then scale everything just-in-case. The ways I've seen this play out: - small apps (few users) just need good schemas and queries - medium apps scale(*) webapp instances that solves client CPU & memory and vertically scale single database writer instance - large apps scale database via sharding or federation/microservices - other apps may not be able to shard a single tenant so also scale via federation/microservices (*) Even if webapps are the CPU/memory bottleneck, overscaling them can make the problem of too many connections to the database for long held transactions. [0] https://github.com/rails/rails/issues/50450 https://github.com/rails/rails/issues/50450
juwfass 2y ago
Optimizing rails is zero sum (it's ruby after all). Do yourself a favor and use pod affinity to schedule rails right next to database while your coworker rewrites it in Elixir/Zig stack.
- citizenpaul 2y ago
  Your comment is confusing. Are you reccomending pod scheduling as a band aid while you rewrite the app in something else? Or making fun of trying to rewrite?
  - juwfass 2y ago
    Forgot to add /s. but yes making fun of both. This is my team, all ex-FAANG.
freedomben 2y ago
I was skeptical given the title that the article could defend it's claim, but the article was strong and made great points. The only thing I would add (which I concur probably doesn't belong in the article itself): Most databases make it trivial to see how long your queries are taking. Use that to help you identify the performance bottlenecks! If a request/response cycle takes 200 ms from the entrypoint of your rails server, and the database query reported (from the db) took 1 ms, that's a whole lot different than if the db query took 150 ms. On scaling in general, in my experience inefficient queries are the vast majority of performance problems for most apps. It's usually fairly easy to identify these as different endpoint will perform very differently so you can get an idea of the baseline and identify aberrations. If that's happening then check your queries. If the whole app (baseline) is slow, TFA has phenonmenal guidance.
hamilyon2 2y ago
There is a certain scale at which this may ring true. This also may by a a tautological statement: when your architecture is right, your tables are well designed out, your queries are optimized to the limit, and you are smart with caching. Of course, now CPU is your problem.
deleted 2y ago
[deleted]
mike_hearn 2y ago
That's basically right except it's worth noting the belief that RDBMS can't scale horizontally is wrong. Most RDBMS engines can't do that, but an example of one that can is Oracle. RAC clusters scale write masters horizontally up to fairly large cluster sizes (e.g. 32 write masters works OK) and obviously read replicas can be added more or less indefinitely. RAC isn't sharding. The Oracle JDBC drivers do support automatic sharding across independent databases or clusters too, but RAC is full horizontal scaling of full SQL including joins. It's also worth noting that if your Ruby app is CPU constrained you could take a look at TruffleRuby. It runs on the JVM, has a full JIT compiler more powerful than YJIT, can deal with native modules, and doesn't have a global interpreter lock (threads do really run in parallel).
- packetlost 2y ago
  This is what I mean when I say Oracle's DB product is actually pretty good. Say what you will about Oracle the company, but their core products are good at their respective domains.
  - mike_hearn 2y ago
    Yeah there's a lot of other neat tricks you can do with it. Someone else in this thread observes that a major issue for most Rails apps is latency. Loading a page does 100 blocking queries serially which means 100 roundtrips (minimum), the latency from that adds up and before you know it your fast computers are taking seconds to render a page. Although it often gets lost inside app frameworks, you can use the Oracle drivers to dispatch every query at once in one request, or even just execute a stored procedure that has all the queries for a page registered inside the DB then work through the results from each query client-side. So you can really optimize the latency heavily if you can structure your app in such a way that it can exploit such things. JDBC supports this for instance but Active Record doesn't.
  - pjmlp 2y ago
    My favourite DBs are Oracle and SQL Server, and no FOSS religion is going to change my mind. Too many folks lose on great technology because of hating the man, 70's hippie style. And yes, I have used Postgres and MySQL and such. It is similar to all the programming languages that have never moved beyond raw command line and basic editor tooling, only better nowadays, because of Microsoft (that they hate) making VSCode (driven by Eric Gamma of Eclipse fame), and LSP a common thing for all workloads.
jerf 2y ago
Another thing I have seen that sneaks up on dynamic scripting language programs is that maybe at first, your code is mostly blocked on database. You prototype something that mostly gets some stuff from the DB and blasts it out on optimized C-based code paths (like JSON encoding) without hardly even looking at it that are fairly fast. But then you need some authentication. And you need authorization, and more deeply than just "yea" or "nay" but doing non-trivial work to determine on some complex resource tree what is and is not permitted. And then you're taking a couple of queries and putting it together with some API call results in scripting code with some non-trivial logic. And then it grows beyond your dev prototype of 10 values to 100,000. And you use a few convenient features of your language without realizing the multiplicative slowdown they impose on your code. You add some automated plugins for turning DB values into these other things without realizing how much that takes. And you add some metadata-driven interpreter action based on what the user is asking for. And one feature at a time, you just keep stacking more and more on. And it can really sneak up on you, but in a mature scripting language codebase it's really easy for it to not just not be blocked on database IO, but for IO to ultimately be such a tiny portion of the runtime that even if the IO was completely free that you'd still have a slow page. Computers are very fast, and very good at what they do, and there's a lot of power to play with in a lot of cases... but it isn't always quite as much as programmers can think it is.
- snovv_crash 2y ago
  I guess this is part of the reason for C++ having zero-cost abstractions.
  - pjmlp 2y ago
    Kind of, the real reason is that C with Classes was Bjarne Stroustoup's "Typescript for C", he wasn't going to use C bare bones after his Simula to BCPL downgrade experience. However to be able to do that, C with Classes abstractions needed to perform just as well as writing raw C.
- swatcoder 2y ago
  You're not really wrong, and what you're talking about is not really related to language choice. What its related to is the narrative you shared in your second paragraph, which I think you were writing mostly for color, perhaps seeming to you as the only or necessary story of how software gets made. "And then", "And then", "And then"... "One feature at a time, you just keep stacking more and more on." There's no foresight. There's no planning. There's no architecture. There's no readiness. There's no preparation. Tomorrow's needs are always beyond the horizon and today's are to be approached strictly on their own terms. If you've taught software development to anyone, you're used to seeing this behavior. Your student doesn't know what they don't know (because you haven't taught them yet), and they're overwhelmed by it, so they just scramble together whatever they can to make it past the next most immediate hurdle. Much (not all) of the industry has basically enshrined that as the way for whole teams and organizations to work now, valorizing it as an expression of agility, nimbleness, and efficiency, and humility. But it inevitably results in code and modules and programs and services and systems where nothing fully snaps together and operational efficiency is lost at every interface, and implementations for new features/challenges need to get awkwardly jammed in as new code rather than elegantly fleshed out as prefigured opportunities. So you're right that most modern projects eventually just become swamped by these inefficiencies, with no ripe ways to resolve them. But it's not because of Rails vs Go or something, it's because pretty much everyone now aspires to build cathedrals without committing to any plans for them ahead of time. What they gain for that is some comfort against FOMO, because they'll never face yesterday's plan conflicting with tomorrow's opportunity (since there is no plan). But they lose a lot too, as you call out very well here.
phamilton 2y ago
Beyond application performance, I've also found a non-trivial drag from the complexity of scaling Rails horizontally. Databases need connection pools, which must be external. Processes vs threads ratio need to be tuned and retuned and the application evolves. Deployments take longer to fully roll out a hundred containers. It's a bunch of tiny bits of complexity that add up. Moving to a fully concurrent runtime (rust, golang, JVM, beam, etc) suddenly makes so much of that no longer a concern.
- pqdbr 2y ago
  For someone that is experienced with Rails but not with the other runtimes mentioned, how do they eliminate the need for db connection pools and speed up deployments across nodes?
  - throw_aw100 2y ago
    I don't have experience with Rails, but I have experience with the other runtimes. I read their comment as needing an external database pool to scale, not that they don't need a pool. When running JVM or the other runtimes, the database pool can be part of the application itself because it uses a different threading model.
    - choilive 2y ago
      Rails also has an application level connection pool for the database but as I understand a connection pooler (pgbouncer et al) between PG and the application servers is still necessary when horizontally scaling.
  - phamilton 2y ago
    To handle 25k qps with Rails, assuming 500ms per request, you need 12k Ruby processes. That probably means you run on 200 boxes with 64 CPUs each. Maybe you get a little bit of oversubscribing of processes to CPUs due to I/O. Conservatively it's still 100 boxes. To handle 25k qps with Rust, you can run 10 boxes. Maybe less. Deployments are complicated (even when we wrap them in simple abstractions). A "rare" delay that impacts 1% of boxes during a deploy will become common on 100 boxes but still somewhat rare on only 10. Running a single rust process per box allows a connection pooler to be in-process. It's more efficient, it supports postgres prepared statements, it's one less operational headache, etc.
hyperman1 2y ago
I have a chain of applications. A postgres spits out data. A java application makes a CSV from it. A apache+php application takes that csv, removes the first column (we don't want to publish the id), then sends it to the destination. Both postgres and java do significant transformations, but the bottleneck for that chain is the PHP code. I already optimized it, and it is now literally 3 lines of code: read line, remove everything before first ',' , write line. This speeds up things enormously from the previous php doing fgetcsv/remove column/fputcsv, but still removing the PHP (keeping only apache ) from the chain doubles the speed of the csv download.
- weaksauce 2y ago
  why does the java application/postgres output the id instead of omitting it in the first place?
  - aetherson 2y ago
    Adding PHP to your stack in order to drop a column from a result is a WILD decision.
    - throw_aw100 2y ago
      I don't want to be a "back seat driver", but it seems strange to me as well. It could be that the original files are used by other processes, and that they for some reason don't want to create two separate files. Maybe an issue with office politics (works on a different team), or an attempt at saving disk space
  - hyperman1 2y ago
    The java thing is an off the shelf application, there is no way to turn off the generated id. So someone slapped some php glue code on it, php being the majority of the code. It is far from optimal, but good enough now to end up low on the priority list. As it runs on windows, I might move it to linux and add sed or something in the chain, losing the php completely. But for now, php it is.
FridgeSeal 2y ago
I’ve seen “oh it’s IO-bound, there’s no point optimising for performance” rolled out so many times, I’m convinced it’s basically lost any intended meaning and as simply a cover-all defence, that many devs don’t appreciate how quick current databases and IO actually is.
- anarazel 2y ago
  The absurd thing about it is that IO bound code IME often is way easier to optimize than CPU bound code. Adding batching or pipelinining is often almost mechanical work and can give huge speedups.
nokun7 2y ago
Great breakdown of the "IO bound Rails app" myth! I really appreciate the focus on profiling and understanding actual application behavior rather than relying on assumptions. The points about threading, database interactions, and context-switch overhead are spot on and offer a fresh perspective on how performance issues are often misunderstood. It's interesting how easy it is to default to surface-level fixes instead of addressing the root cause. Have you thought about adding a case study or some benchmarks for common workloads? It could really help teams put these ideas into practice.
- byroot 2y ago
  The lobste.rs maintainer kindly offered to collect data in production: https://github.com/lobsters/lobsters/pull/1442 https://github.com/lobsters/lobsters/pull/1442 I haven't taken the time to fully analyze the data, but surface level show lobster.rs is indeed ~40% IO, as suspected: https://gist.github.com/byroot/e1fcfd1f8e172f5d9c5bfecefebdb15f https://gist.github.com/byroot/e1fcfd1f8e172f5d9c5bfecefebdb...
alkonaut 2y ago
The article gives an example of how apps are usually benchmarked with log queries. But in a real world scenario wouldn't you just analyze what the app is actually doing and see whether what fraction it actually waits for IO, waits for sync primitives, or something else? Why do logging for this? It's the performance equivalent of printf debugging.
- byroot 2y ago
  I use simple logging code to show how the data is collected. In a real production setup, that timing would be reported into Datadog / NewRelic or whatever APM service, but collected in a similar way.
  - alkonaut 2y ago
    Sure but if you look at that logging data which you might be collecting regardless, adn you see anything that in the slightest way suggests a performance issue, or if you look at your hosting bill and wonder where the money is going, or if you need fast CPUs or fast disks etc, just having good recorded perf data would be one of the first things you'd do?
    - byroot 2y ago
      I guess I really don't understand what you are trying to say.
      - alkonaut 2y ago
        I'm saying you'd just record perf stats with a normal profiler and never do any sort of decision or analysis based on what you read in a log?
pjmlp 2y ago
Fully spot on, that is why using Tcl with lots of native extensions might have been a great solution in the late 90's dotcom wave, we quickly validated with our rewrite into .NET, still beta at the time, the huge difference that having a JIT in the box makes. Hence why afterwards I never been a fan of any language for deployment production without AOT or JIT tooling on the reference implementation.

227 points by thunderbong 2y ago

Very nice discussion. A lot of times I see people assuming that Web Apps are IO bound anyway to justify technical decisions. Nevertheless here am I looking at our stats: 60% is python, 40% is IO and Postgres is getting faster than Python at a faster pace.

gtirloni 2y ago

I didn't get the point about YJIT only helping 15-30% in real apps while IO-less benchmarks see 2-3x. It seems like a counterpoint to the main argument of the article. Instead of guessing if the DB is performing badly by looking at Rails, why not go directly to the DB?

byroot 2y ago
Someone else pointed the same thing on bluesky, so I guess this part of my post isn't very clear. First, I/O bound isn't necessarily a very well defined term, but generally speaking, it implies that you can't substantially speedup such system by speeding up code execution. The fact that YJIT did yield substantial performance improvement in the real world suggest many Rails apps aren't in fact strictly I/O bound. Now about the 15-30% vs 2-3x, what I mean by that is the benchmark where YJIT yield this much are mostly micro-benchmarks, on more complex and heterogenous code, the gains are much smaller, hence we can assume the "Ruby" part of these applications wasn't improved by this much. So for YJIT to be able to yield `15-30%` latency gains, this latency must have been in large part composed of Ruby execution. One last thing to note is that YJIT can only speedup pure Ruby code, in a Rails application, a large part of the CPU time isn't in pure Ruby code, but in various methods implemented in C that YJIT can't speedup. Ultimately the lobste.rs maintainer kindly offered to run an experiment in production, so we should soon see if my assumptions hold true or if I was way off, at least for that particular app: https://github.com/lobsters/lobsters/pull/1442 https://github.com/lobsters/lobsters/pull/1442 Edit: > Instead of guessing if the DB is performing badly by looking at Rails That isn't really the topic though. The question is more about how much Ruby's GVL is actually released, and its implications.
SkiFire13 2y ago
The point was that 15-30% is a very big speedup for a change that impacts what is supposed to take a very small part of the total time. See also Amdahl's law [1] Suppose that the CPU-intensive part of the program was actually speed up by 3x. This means that previously it took 1.5x as much time as the one that was saved, so thoughly on the order of 22-45% of the total time of your program. In practice the speedup was probably lower than 3x, which means the 22-45% figure would be even higher. These are really really high number for a system that was supposed to spend most of the time (>90%) blocked in IO (which was not speed up by YJIT). For comparison, if the CPU-intensive part actually took 10% of the time, then 10% would be maximum speedup you could get (which is way lower than 15-30%) and a 3x speedup would only result in a ~6% speedup. [1]: https://en.m.wikipedia.org/wiki/Amdahl%27s_law https://en.m.wikipedia.org/wiki/Amdahl%27s_law
ksec 2y ago
If your Application is 50/50 in CPU / IO at a 100ms request. Getting 30% Reduction in total response time would mean the request is now only 70ms, DB stays the same at 50ms, your CPU is now only 20ms. That is 2.5x speed up compare to the 50ms previously. Exactly inline with the 2-3x IO-less benchmarks.

rf15 2y ago

Usually it's the inefficiencies of architectural decisions (your algos, layouts, etc.) you put into that rails app.

scosman 2y ago
Sometimes. But if you have any real processing to do, it can be hard/impossible to do it quickly in ruby. You end up writing c extensions, or another app server in another language for the hard stuff.
ndriscoll 2y ago
Right. Simple test: check the throughput your database can do with bulk queries. Do a single query select of an xml or json struct that outputs the page rails would return (so one rendered page per row). Select enough rows for the query to take 1-10s to get a sense for rendered pages/second. Do the same for inserts/updates. If you're not close to what postgres alone can do with bulk queries, you're obviously not IO bound. Other sanity check: nvme drives are ~10000x faster than hard disks were. Are you doing 10000x more useful disk IO than people were doing with hard disks?
- anarazel 2y ago
  For typical rails like applications that are "bottlenecked on the database", latency is a much bigger factor than actual query processing performance. Even if careful about placement of the database and "application servers". Just as an example, here's the results of pgbench -S of a single client, with pgbench running on different servers (a single pkey lookup): pgbench on database server: 36233 QPS pgbench on a different system (local 10gbit network): 2860 QPS It's rare to actually have that low latency to the database in the real world IME. If the same pkey lookup is executed utilizing pipelining, I get ~145k QPS from both locally and remotely.
  - ndriscoll 2y ago
    That's still an architectural thing. If CPU isn't an issue, why would you be running your application server on a different machine from your database? Connecting to a unix domain socket will give better latency and throughput and security as long as you can scale vertically. Even if you have it on another machine, it's on the same switch, right (also an architectural choice)? So latency should be sub millisecond, and irrelevant for a single request. So we assume we're talking about many requests, but then why not combine/batch your queries (also an architectural choice)? Now latency (and other things like database latches and fsyncs. Very important.) is amortized and again irrelevant. You should be able to hit close to the pure pg bulk query case (and e.g. a rust application server can do this in practice for me with a 4 core machine and an old SATA SSD with ~50k read IOPS). Point is, the optimal case for building a web page is probably to do pg queries that return the xml of the page (letting you do subselects to combine everything into one query). So see how fast that is, and if your application can't match that, the database is not your bottleneck.

scosman 2y ago

“CPU starvation looks like IO to most eyes” Yes! And add in that you are holding a database connection for longer than needed, putting scaling pressure on your DB, which is the harder thing to scale.

PaulHoule 2y ago
I'm not sure if it is the same phenomenon but I was really disappointed with the performance of a python aiohttp server [1] when I went from the application serving an html page with maybe 2-3 small images that would stick in the cache (icons) and instead was serving 20-50 photographic images per page, particularly over the wrong direction of an ADSL connection. I thought this was low CPU but it was not low CPU enough, in the end I switched to gunicorn (running in WSL) with IIS fronting and serving the images (woulda been nginix in a pure-Linux environment) I think the tiny amount of CPU to serve the images was still getting blocked by requests that used a little more CPU. With gunicorn I'm not afraid to add tasks that crunch on the CPU a little harder too. [1] built a lot of mini-apps that worked great before
- CoolCold 2y ago
  Everything is better with Nginx! As sysadmin I've seen many cases when not letting backend to serve static files drops latency and load significantly. In the worst case, X-accel-redirect is still better than serving through most of the frameworks.
WJW 2y ago
Having the connection for longer than needed only puts more pressure on databases on which connections are a scarce resource (basically only postgres, and even then it's easily fixed with pgbouncer). When using a database with a better connection model, holding a connection too long will actually decrease total load on the database because each connection is idle for a larger percentage of the time.
- CoolCold 2y ago
  Isn't pgbouncer putting extra limitations like prepared statemens use ? Genuine question

Azerty9999 2y ago

There is a highly relevant follow-up post by the author here, too: https://byroot.github.io/ruby/performance/2025/01/23/io-instrumentation.html https://byroot.github.io/ruby/performance/2025/01/23/io-inst...

jupp0r 2y ago

When people say these apps are IO bound, what they actually mean is that they are memory bound. While Rails is waiting for IO due to its lack of concurrency support and the ubiquitous use of global variables in the ecosystem, a passenger worker is taking up >300mb of memory for a medium sized Rails app. Server memory limits the amount of workers you can have waiting like that, which in turn limits overall throughput per server.

caseyohara 2y ago
That you mention Passenger makes me wonder how outdated your comment is. The most recent Rails community survey reported only 13% using Passenger. https://railsdeveloper.com/survey/2024/ https://railsdeveloper.com/survey/2024/
byroot 2y ago
That may be true for very memory constrained platforms like Heroku, but is a non issue at larger scale. Taking your figure of 300MiB per process, and assuming you might want to run 1.5 process per core to account for IO-wait, that's 450MiB per core (ignoring copy-on-write). If you look at various hosting offering, most offer something like 4GiB of RAM per core (e.g. EC2 "general purpose" instances). As for the lack of concurrency support, Active Record has asynchronous queries since a few versions, and is now even async/fiber compatible. So if you truly have an app that would benefit from more concurrency, you can do it.
pydry 2y ago
No, it just means that they spend most of their time waiting around for database and network queries to finish.
marcosdumay 2y ago
They mean the app takes data from one place, does a trivial computation, and pushed it at another place. So pushing data around is all the application does. And they don't know both such places are actually faster than the trivial computation because their code runs way slower than the ones at the origin and destination.

karmakaze 2y ago

The discussion in the referenced Rails issue "Set a new default for the Puma thread count"[0] to me is much more telling of IO vs CPU than this simplified post. There are many assumptions here especially not considering anything about the database itself. The Rails issue also considers benchmarks which when benchmarking Rails would configure the database to not be the bottleneck. That's not true of real systems. The advice to use async queries for an IO-bound app could backfire if the reason the queries are slow is because the database is overloaded--adding concurrent queries only increases its thrashing and latency. The best thing to do is consider the whole system. Don't throw the everything that's not the Rails app as IO. Is it actually doing network IO, or is it CPU, memory, or IO bound in the database? Maybe its not even a lack of CPU proper on the database but that it's being wasted on write contention/locks. Only then will you be able to choose the right course. Another way to go is blindly try different configurations and use what works well without full understanding, which is fine until you have an outage you can't explain then scale everything just-in-case. The ways I've seen this play out: - small apps (few users) just need good schemas and queries - medium apps scale(*) webapp instances that solves client CPU & memory and vertically scale single database writer instance - large apps scale database via sharding or federation/microservices - other apps may not be able to shard a single tenant so also scale via federation/microservices (*) Even if webapps are the CPU/memory bottleneck, overscaling them can make the problem of too many connections to the database for long held transactions. [0] https://github.com/rails/rails/issues/50450 https://github.com/rails/rails/issues/50450

juwfass 2y ago

Optimizing rails is zero sum (it's ruby after all). Do yourself a favor and use pod affinity to schedule rails right next to database while your coworker rewrites it in Elixir/Zig stack.

citizenpaul 2y ago
Your comment is confusing. Are you reccomending pod scheduling as a band aid while you rewrite the app in something else? Or making fun of trying to rewrite?
- juwfass 2y ago
  Forgot to add /s. but yes making fun of both. This is my team, all ex-FAANG.

freedomben 2y ago

I was skeptical given the title that the article could defend it's claim, but the article was strong and made great points. The only thing I would add (which I concur probably doesn't belong in the article itself): Most databases make it trivial to see how long your queries are taking. Use that to help you identify the performance bottlenecks! If a request/response cycle takes 200 ms from the entrypoint of your rails server, and the database query reported (from the db) took 1 ms, that's a whole lot different than if the db query took 150 ms. On scaling in general, in my experience inefficient queries are the vast majority of performance problems for most apps. It's usually fairly easy to identify these as different endpoint will perform very differently so you can get an idea of the baseline and identify aberrations. If that's happening then check your queries. If the whole app (baseline) is slow, TFA has phenonmenal guidance.

hamilyon2 2y ago

There is a certain scale at which this may ring true. This also may by a a tautological statement: when your architecture is right, your tables are well designed out, your queries are optimized to the limit, and you are smart with caching. Of course, now CPU is your problem.

deleted 2y ago

[deleted]

mike_hearn 2y ago

That's basically right except it's worth noting the belief that RDBMS can't scale horizontally is wrong. Most RDBMS engines can't do that, but an example of one that can is Oracle. RAC clusters scale write masters horizontally up to fairly large cluster sizes (e.g. 32 write masters works OK) and obviously read replicas can be added more or less indefinitely. RAC isn't sharding. The Oracle JDBC drivers do support automatic sharding across independent databases or clusters too, but RAC is full horizontal scaling of full SQL including joins. It's also worth noting that if your Ruby app is CPU constrained you could take a look at TruffleRuby. It runs on the JVM, has a full JIT compiler more powerful than YJIT, can deal with native modules, and doesn't have a global interpreter lock (threads do really run in parallel).

packetlost 2y ago
This is what I mean when I say Oracle's DB product is actually pretty good. Say what you will about Oracle the company, but their core products are good at their respective domains.
- mike_hearn 2y ago
  Yeah there's a lot of other neat tricks you can do with it. Someone else in this thread observes that a major issue for most Rails apps is latency. Loading a page does 100 blocking queries serially which means 100 roundtrips (minimum), the latency from that adds up and before you know it your fast computers are taking seconds to render a page. Although it often gets lost inside app frameworks, you can use the Oracle drivers to dispatch every query at once in one request, or even just execute a stored procedure that has all the queries for a page registered inside the DB then work through the results from each query client-side. So you can really optimize the latency heavily if you can structure your app in such a way that it can exploit such things. JDBC supports this for instance but Active Record doesn't.
- pjmlp 2y ago
  My favourite DBs are Oracle and SQL Server, and no FOSS religion is going to change my mind. Too many folks lose on great technology because of hating the man, 70's hippie style. And yes, I have used Postgres and MySQL and such. It is similar to all the programming languages that have never moved beyond raw command line and basic editor tooling, only better nowadays, because of Microsoft (that they hate) making VSCode (driven by Eric Gamma of Eclipse fame), and LSP a common thing for all workloads.

jerf 2y ago

Another thing I have seen that sneaks up on dynamic scripting language programs is that maybe at first, your code is mostly blocked on database. You prototype something that mostly gets some stuff from the DB and blasts it out on optimized C-based code paths (like JSON encoding) without hardly even looking at it that are fairly fast. But then you need some authentication. And you need authorization, and more deeply than just "yea" or "nay" but doing non-trivial work to determine on some complex resource tree what is and is not permitted. And then you're taking a couple of queries and putting it together with some API call results in scripting code with some non-trivial logic. And then it grows beyond your dev prototype of 10 values to 100,000. And you use a few convenient features of your language without realizing the multiplicative slowdown they impose on your code. You add some automated plugins for turning DB values into these other things without realizing how much that takes. And you add some metadata-driven interpreter action based on what the user is asking for. And one feature at a time, you just keep stacking more and more on. And it can really sneak up on you, but in a mature scripting language codebase it's really easy for it to not just not be blocked on database IO, but for IO to ultimately be such a tiny portion of the runtime that even if the IO was completely free that you'd still have a slow page. Computers are very fast, and very good at what they do, and there's a lot of power to play with in a lot of cases... but it isn't always quite as much as programmers can think it is.

snovv_crash 2y ago
I guess this is part of the reason for C++ having zero-cost abstractions.
- pjmlp 2y ago
  Kind of, the real reason is that C with Classes was Bjarne Stroustoup's "Typescript for C", he wasn't going to use C bare bones after his Simula to BCPL downgrade experience. However to be able to do that, C with Classes abstractions needed to perform just as well as writing raw C.
swatcoder 2y ago
You're not really wrong, and what you're talking about is not really related to language choice. What its related to is the narrative you shared in your second paragraph, which I think you were writing mostly for color, perhaps seeming to you as the only or necessary story of how software gets made. "And then", "And then", "And then"... "One feature at a time, you just keep stacking more and more on." There's no foresight. There's no planning. There's no architecture. There's no readiness. There's no preparation. Tomorrow's needs are always beyond the horizon and today's are to be approached strictly on their own terms. If you've taught software development to anyone, you're used to seeing this behavior. Your student doesn't know what they don't know (because you haven't taught them yet), and they're overwhelmed by it, so they just scramble together whatever they can to make it past the next most immediate hurdle. Much (not all) of the industry has basically enshrined that as the way for whole teams and organizations to work now, valorizing it as an expression of agility, nimbleness, and efficiency, and humility. But it inevitably results in code and modules and programs and services and systems where nothing fully snaps together and operational efficiency is lost at every interface, and implementations for new features/challenges need to get awkwardly jammed in as new code rather than elegantly fleshed out as prefigured opportunities. So you're right that most modern projects eventually just become swamped by these inefficiencies, with no ripe ways to resolve them. But it's not because of Rails vs Go or something, it's because pretty much everyone now aspires to build cathedrals without committing to any plans for them ahead of time. What they gain for that is some comfort against FOMO, because they'll never face yesterday's plan conflicting with tomorrow's opportunity (since there is no plan). But they lose a lot too, as you call out very well here.

phamilton 2y ago

Beyond application performance, I've also found a non-trivial drag from the complexity of scaling Rails horizontally. Databases need connection pools, which must be external. Processes vs threads ratio need to be tuned and retuned and the application evolves. Deployments take longer to fully roll out a hundred containers. It's a bunch of tiny bits of complexity that add up. Moving to a fully concurrent runtime (rust, golang, JVM, beam, etc) suddenly makes so much of that no longer a concern.

pqdbr 2y ago
For someone that is experienced with Rails but not with the other runtimes mentioned, how do they eliminate the need for db connection pools and speed up deployments across nodes?
- throw_aw100 2y ago
  I don't have experience with Rails, but I have experience with the other runtimes. I read their comment as needing an external database pool to scale, not that they don't need a pool. When running JVM or the other runtimes, the database pool can be part of the application itself because it uses a different threading model.
  - choilive 2y ago
    Rails also has an application level connection pool for the database but as I understand a connection pooler (pgbouncer et al) between PG and the application servers is still necessary when horizontally scaling.
- phamilton 2y ago
  To handle 25k qps with Rails, assuming 500ms per request, you need 12k Ruby processes. That probably means you run on 200 boxes with 64 CPUs each. Maybe you get a little bit of oversubscribing of processes to CPUs due to I/O. Conservatively it's still 100 boxes. To handle 25k qps with Rust, you can run 10 boxes. Maybe less. Deployments are complicated (even when we wrap them in simple abstractions). A "rare" delay that impacts 1% of boxes during a deploy will become common on 100 boxes but still somewhat rare on only 10. Running a single rust process per box allows a connection pooler to be in-process. It's more efficient, it supports postgres prepared statements, it's one less operational headache, etc.

hyperman1 2y ago

I have a chain of applications. A postgres spits out data. A java application makes a CSV from it. A apache+php application takes that csv, removes the first column (we don't want to publish the id), then sends it to the destination. Both postgres and java do significant transformations, but the bottleneck for that chain is the PHP code. I already optimized it, and it is now literally 3 lines of code: read line, remove everything before first ',' , write line. This speeds up things enormously from the previous php doing fgetcsv/remove column/fputcsv, but still removing the PHP (keeping only apache ) from the chain doubles the speed of the csv download.

weaksauce 2y ago
why does the java application/postgres output the id instead of omitting it in the first place?
- aetherson 2y ago
  Adding PHP to your stack in order to drop a column from a result is a WILD decision.
  - throw_aw100 2y ago
    I don't want to be a "back seat driver", but it seems strange to me as well. It could be that the original files are used by other processes, and that they for some reason don't want to create two separate files. Maybe an issue with office politics (works on a different team), or an attempt at saving disk space
- hyperman1 2y ago
  The java thing is an off the shelf application, there is no way to turn off the generated id. So someone slapped some php glue code on it, php being the majority of the code. It is far from optimal, but good enough now to end up low on the priority list. As it runs on windows, I might move it to linux and add sed or something in the chain, losing the php completely. But for now, php it is.

FridgeSeal 2y ago

I’ve seen “oh it’s IO-bound, there’s no point optimising for performance” rolled out so many times, I’m convinced it’s basically lost any intended meaning and as simply a cover-all defence, that many devs don’t appreciate how quick current databases and IO actually is.

anarazel 2y ago
The absurd thing about it is that IO bound code IME often is way easier to optimize than CPU bound code. Adding batching or pipelinining is often almost mechanical work and can give huge speedups.

nokun7 2y ago

Great breakdown of the "IO bound Rails app" myth! I really appreciate the focus on profiling and understanding actual application behavior rather than relying on assumptions. The points about threading, database interactions, and context-switch overhead are spot on and offer a fresh perspective on how performance issues are often misunderstood. It's interesting how easy it is to default to surface-level fixes instead of addressing the root cause. Have you thought about adding a case study or some benchmarks for common workloads? It could really help teams put these ideas into practice.

byroot 2y ago
The lobste.rs maintainer kindly offered to collect data in production: https://github.com/lobsters/lobsters/pull/1442 https://github.com/lobsters/lobsters/pull/1442 I haven't taken the time to fully analyze the data, but surface level show lobster.rs is indeed ~40% IO, as suspected: https://gist.github.com/byroot/e1fcfd1f8e172f5d9c5bfecefebdb15f https://gist.github.com/byroot/e1fcfd1f8e172f5d9c5bfecefebdb...

alkonaut 2y ago

The article gives an example of how apps are usually benchmarked with log queries. But in a real world scenario wouldn't you just analyze what the app is actually doing and see whether what fraction it actually waits for IO, waits for sync primitives, or something else? Why do logging for this? It's the performance equivalent of printf debugging.

byroot 2y ago
I use simple logging code to show how the data is collected. In a real production setup, that timing would be reported into Datadog / NewRelic or whatever APM service, but collected in a similar way.
- alkonaut 2y ago
  Sure but if you look at that logging data which you might be collecting regardless, adn you see anything that in the slightest way suggests a performance issue, or if you look at your hosting bill and wonder where the money is going, or if you need fast CPUs or fast disks etc, just having good recorded perf data would be one of the first things you'd do?
  - byroot 2y ago
    I guess I really don't understand what you are trying to say.
    - alkonaut 2y ago
      I'm saying you'd just record perf stats with a normal profiler and never do any sort of decision or analysis based on what you read in a log?

pjmlp 2y ago

Fully spot on, that is why using Tcl with lots of native extensions might have been a great solution in the late 90's dotcom wave, we quickly validated with our rewrite into .NET, still beta at the time, the huge difference that having a JIT in the box makes. Hence why afterwards I never been a fan of any language for deployment production without AOT or JIT tooling on the reference implementation.