4 ms·

Yeah. I mentioned that in the final test where I write to a temporary file, then rename. This is much slower, probably due to an implicit fsync.

electroly 5y ago
You can disable the fsync calls in sqlite if you wanted to do a little better with this benchmark. You're also explicitly choosing the WAL in your go-sqlite3 configuration which is not at all replicated by your filesystem test. I think, honestly, that you're just going to mislead and confuse people who don't know any better with this writeup. I can write faster to /dev/null, too, but that isn't a very interesting comparison.
- HideousKojima 5y ago
  Does /dev/null support sharding?
  - pa7ch 5y ago
    Yes.
  - grogers 5y ago
    Even better, it's embarrassingly parallel!
cafxx 5y ago
I am not aware of implicit fsyncs. Can you please link to what you are referring to?
- prussian 5y ago
  Probably in reference to Ext4's[0] configurable rename-replace behavior under auto_da_alloc [0]: https://www.kernel.org/doc/Documentation/filesystems/ext4.txt https://www.kernel.org/doc/Documentation/filesystems/ext4.tx... Edit: note this isn't a universal property either, so it's still wrong: https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_crash_guarantees_of_overwrite-by-rename.3F https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_cra...
  - cafxx 5y ago
    Yeah considering that the test is on btrfs, I would really be interested in knowing what OP is referring to. Also, even if we were on ext4, it's worth pointing out that the ext4 documentation refers to apps that rely on that behavior as "broken applications", so it's hardly a good term of comparison.
lamontcg 5y ago
There's some 10-year old webpage that used to be popular on HN that goes into data integrity issues and fsync and filesystem databases that someone should dig up so you can read.
- nyanpasu64 5y ago
  https://danluu.com/file-consistency/ https://danluu.com/file-consistency/, https://stackoverflow.com/a/41362774 https://stackoverflow.com/a/41362774
  - lamontcg 5y ago
    Yes that's the one. And damn near every application is "broken" according to the ext4 documentation.

electroly 5y ago

You can disable the fsync calls in sqlite if you wanted to do a little better with this benchmark. You're also explicitly choosing the WAL in your go-sqlite3 configuration which is not at all replicated by your filesystem test. I think, honestly, that you're just going to mislead and confuse people who don't know any better with this writeup. I can write faster to /dev/null, too, but that isn't a very interesting comparison.

HideousKojima 5y ago
Does /dev/null support sharding?
- pa7ch 5y ago
  Yes.
- grogers 5y ago
  Even better, it's embarrassingly parallel!

cafxx 5y ago

I am not aware of implicit fsyncs. Can you please link to what you are referring to?

prussian 5y ago
Probably in reference to Ext4's[0] configurable rename-replace behavior under auto_da_alloc [0]: https://www.kernel.org/doc/Documentation/filesystems/ext4.txt https://www.kernel.org/doc/Documentation/filesystems/ext4.tx... Edit: note this isn't a universal property either, so it's still wrong: https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_crash_guarantees_of_overwrite-by-rename.3F https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_cra...
- cafxx 5y ago
  Yeah considering that the test is on btrfs, I would really be interested in knowing what OP is referring to. Also, even if we were on ext4, it's worth pointing out that the ext4 documentation refers to apps that rely on that behavior as "broken applications", so it's hardly a good term of comparison.

lamontcg 5y ago

There's some 10-year old webpage that used to be popular on HN that goes into data integrity issues and fsync and filesystem databases that someone should dig up so you can read.

nyanpasu64 5y ago
https://danluu.com/file-consistency/ https://danluu.com/file-consistency/, https://stackoverflow.com/a/41362774 https://stackoverflow.com/a/41362774
- lamontcg 5y ago
  Yes that's the one. And damn near every application is "broken" according to the ext4 documentation.