3 ms·
Interesting that the recommended default for performance is "worker" instead of "io_uring". The early posts about async IO in Postgres 18 read a bit different h
by fabian2k 1y ago
Interesting that the recommended default for performance is "worker" instead of "io_uring". The early posts about async IO in Postgres 18 read a bit different here, this post here appears more authorative to me though. We'll see how this turns out in future releases.
At least it is only two settings to look at for tuning, though it does seem this is the kind of setting you have to try out with your own DB and workload.
Right now async IO is used for sequential scans and bitmap scans, not for index scans. My initial guess would be that it mostly helps for complex queries (that use multiple indexes, so bitmap scans) and unoptimized queries (sequential scans), not so much for straightforward and/or optimized queries that use a single index. But this is really just a guess, I'm way out of my depth at this point. I'm curious how much it'll help once it is implemented for index scans as well.
- nromiun 1y agoThe article explains this: > The default is io_method = worker. We did consider defaulting both to sync or io_uring, but I think worker is the right choice. It’s actually “asynchronous”, and it’s available everywhere (because it’s our implementation). I think a pool of workers is a sensible default. Many environments disable io_uring for security reasons anyway.
- fabian2k 1y agoSorry, I was ambiguous in my statement there. I'm not surprised it's the default, that makes sense for all the reasons you state. I'm surprised this post also recommends "worker" instead of "io_uring" for performance in general. That is different from the impression I got based on earlier blog posts before Postgres 18 was released.
- pgaddict 1y agoI did a lot of tests comparing the io_method choices, and I'm yet to see a realistic query where it makes a significant difference of more than a couple percent (in either direction). I'm sure it's possible to construct such queries, and it's interesting, but for real workloads it's mostly not noticeable. At least that's how I see it right now, we'll see how that works on a much wider range of hardware and systems. The github repo linked from the pgsql-hackers post has a lot more results, some charts include results for the index prefetching patch - and there it makes more difference in some cases. But the patch is still fairly rough, it could be a bug in it too, and it changed a lot since August.
- film42 1y agoMy guess is both will look about the same with real world workloads. Worker is certainly more predictable which is safer in general. That said, I appreciate the callout about signal throughput on workers (fewer connections farm to more processes vs each connection getting its own io_uring setup with upper bound being the throughput for a single process). Again, I doubt it makes any difference for 99.9999% of apps out there.
- taeric 1y agoMy gut is that the recommendation flows from the idea of a portable solution with more controls? That is, io_uring is limited in both target deployments, and in what you can do to adjust performance considerations. The first is somewhat obvious, with how new io_uring is and how it is specifically a linux thing. I don't think that is necessarily bad. My guess is you can get really good performance with relatively little code compared to other options. The second, though, is a bit tougher to consider. For one, it will be more code. Managing workers is managing code that you probably don't have on your radar. That said, you have full control over the worker so that you can make different priority work queues in ways that I don't see how you could do with io_uring.
- pgaddict 1y ago> Right now async IO is used for sequential scans and bitmap scans, not for index scans. My initial guess would be that it mostly helps for complex queries (that use multiple indexes, so bitmap scans) and unoptimized queries (sequential scans), not so much for straightforward and/or optimized queries that use a single index. But this is really just a guess, I'm way out of my depth at this point. I'm curious how much it'll help once it is implemented for index scans as well. Those are good guesses, IMHO. For sequential scans, some of the "async" work could be done by kernel read-ahead, but AIO makes it explicit and loads the data into shared buffers, not just page cache. For bitmap scans we already had prefetching by fadvise, which is somewhat similar to read-ahead (also into page cache), and there were some bugs that made it ineffective in some cases, and AIO fixes that. For index scans the difference can be an order of magnitude (say, 5-10x). Doing random I/O block by block is simply awful, prefetching data is important. I was just doing some testing on TPC-H, and on scale 50 I see Q8 going from 130s to 20s, and Q19 from 50s to 8s. And smaller improvements for a couple more queries. Of course, it depends on what else the query is doing - if it's spending 1% on the index, you won't notice a difference.