3 ms·
I became curious how fast fdatasync (a better version of fsync) is, and asked LLM to generate a microbenchmark for me. The benchamrk accepts records size, numbe
by codedokode 2mo ago
I became curious how fast fdatasync (a better version of fsync) is, and asked LLM to generate a microbenchmark for me. The benchamrk accepts records size, number of records, opens a file, and appends records one by one, making fdatasync after each one. On my SSD, the throughput is 120 fsyncs/second, or ~0.5 MB/s. So if I had Redis to flush the log after each operation, it wouldn't be able to serve those thousands of connections. That's why Redis is either used without AOF, or they flush the data once per second. And that's why SQL databases might look "slow".
Of course one could optimize this - for example, while fsync is being executed, we could accept the queries from other clients and execute them, and once previous flush finishes, flush multiple transactions at once. However, I am not sure if Redis can do this due to being single-thread.
And obviously maybe there are problems with drivers, or with my consumer-level SSD and maybe "professional" SSDs can do more flushes per second.
Writing the code took less than a minute. I often do microbenchmarks now because it is so easy.
- deleted 2mo ago[deleted]
- deleted 2mo ago[deleted]
- deleted 2mo ago[deleted]
- jdw64 2mo agoConcurrent connection count and durable write throughput are different things. And AOF doesn't necessarily mean one fsync per record. The benchmark is taking the worst-case speed of Redis and comparing it to a best-case SQL benchmark. Redis AOF isn't one fsync per record—Redis can use group commit to share fsync across multiple commands. In other words, that could mean a difference of tens of times in your benchmark. I see it differently.
- CoolCold 2mo agomy common dumb test approach with fio fio --rw=write --ioengine=sync --fdatasync=1 --directory=test-data --size=220m --bs=2300 --name=mytest Consumer grade SSDs 100-500 iops, datacenter SSDs in thousands (like ~ 5 year old samsung model on Hetzher AFAIR had ~ 3000 iops in that test). Cheap VPSes - easily in 5-30 iops of this sort more often then not, fsync/fdatasync is overlooked by programmers