2 ms·
> Many existing filesystems simply cannot support such an API because they're not journaling or CoW. Thats fine. Legacy filesystems can just use a best-effort
by josephg 6y ago
> Many existing filesystems simply cannot support such an API because they're not journaling or CoW.
Thats fine. Legacy filesystems can just use a best-effort approach like they do now with 95% of programs. I already wouldn't put data I care about on ext2. Programs could detect transaction support with a syscall then decide whether to fallback to old behaviour, lose reliability or for a database maybe just refuse to run entirely.
> This suddenly looks more and more like a database than a posix filesystem.
POSIX filesystem apis aren't fit for use when you care about correctness. And as I understand it ZFS, btrfs and APFS already look more like databases internally than traditional posix filesystems. If that ship is sailing anyway we may as well get all the benefits we can.
> There are many use-cases that don't need such hard guarantees and want throughput, minimal overhead and latency instead.
Are there? I can't think of many. Maybe copying big files? We'll need to keep supporting the current unsafe-by-default posix API forever for current software anyway. I just don't think it should be the default for 95% of applications. When I do a git commit, its plenty fast enough. But I definitely don't want my repository to get corrupted if my computer crashes.
If the recommendation is that everything that matters should be in a sqlite database because posix is so hard to use, well, that sounds like we agree more than we disagree. It seems like a pity though, and I think the filesystem is worth salvaging.
And with the API I'm suggesting, I don't think large operations need to get much slower. The transactional API could just optimistically write at full speed to unlinked sectors on disk, then atomically update pointers to reference the new sectors when the transaction is committed. That wouldn't add much amortized overhead. I'm sure there's more complexity than that in practice, but it sounds pretty tractable and fast in the common case of a single writer. (And in the slow case where multiple programs try to fight over the same file, slow would be an improvement over the current behaviour of "probably everything breaks".)
> I think the postgres/fsync error saga raised awareness of that and there were some proposals how to provide a better error channel to userspace.
Its still almost impossible to write a database correctly on top of the POSIX APIs. Especially if you want it to be correct. That seems fixable. We should fix it.
- the8472 6y ago> If the recommendation is that everything that matters should be in a sqlite database because posix is so hard to use, well, that sounds like we agree more than we disagree. I don't mean everyone should use sqlite. We just need more safe abstractions over posix APIs that cover different use-cases. If you need to concurrently query and modify many small records, then sure, go with sqlite. > Are there? I can't think of many. Maybe copying big files? It's not just the big files. Millions of small files would cause some overhead if you actually tried commiting them instead of relying on best-effort behavior (see git case below). Temp and cache directories. Archival. Compiler artifacts. Often you don't even want them to actually hit the disk at all if you have enough RAM. A file can act as nothing more than named swap space. > When I do a git commit, its plenty fast enough. But I definitely don't want my repository to get corrupted if my computer crashes. Sure, that's for the canonical git storage, which rarely touches many files anyway. But if you checkout a worktree (e.g. by switching branches) then putting all the thousands of files from the .git object storage into the work tree does not need durability. You want to be able to work on those files as soon as possible, even if they're just in the page cache and not written to disk yet. The command shouldn't block on fsync.
- josephg 6y ago> We just need more safe abstractions over posix APIs that cover different use-cases. I reach for lmdb for a lot of small projects. It feels like failure though. We shouldn't need things like that just to deterministically save data to disk. > The command shouldn't block on fsync. Sure - some operations can and should discard durability for higher performance. Sounds great to me. But maybe you should opt in to that. It certainly shouldn't be the only behaviour the kernel provides. If Inkscape crashes halfway through saving my work, it shouldn't by default end up in a half-saved state where my file is permanently corrupted. But we can have our cake and eat it too here. It can be as simple as, if you don't wrap your write() calls in a filesystem transaction, the behaviour stays the same as it always was. Or maybe the default behaviour is that a write call entirely happens or entirely doesn't happen, but you can optionally ioctl(NO_DURABILITY) or something for extra performance in git checkout. I hear you that git checkout performance (and things like it) are important to you. Keeping the current performance of those operations sounds good to me too!