3 ms·
> If I/O was the bottleneck, parallelizing it won't help, your SSD/network link/database won't get magically faster. Of course it will. Near every serious DB w
by ilyt 4y ago
> If I/O was the bottleneck, parallelizing it won't help, your SSD/network link/database won't get magically faster.
Of course it will. Near every serious DB will allow to work on multiple requests in parallel and unless the DB itself is on something that's slow you will get data faster from 2 parallel requests than from serializing them
NVMe SSDs in particular can easily fill what single thread can read, just run fio with single vs parallel threads to see that.
> If you're writing to files, most of the time the parallism will be hard to implement correctly. SQLite doesn't support parallel writes for example.
That's just one random example. If all you do is "read data, parse ,write data" in some batch job you can have massive parallelism. Sharding is also easy way to fill up the IO.