4 ms·

> All filesystem operations on modern computers should be atomic Well that seems a bit over the top. You would pay the price of full consistency for every writ

> All filesystem operations on modern computers should be atomic Well that seems a bit over the top. You would pay the price of full consistency for every write, while just one write every 3 years will fail. It would be insanely slow and wasteful. Not to mention, that would basically disallow any kind of OS or hardware caching, which is immensely important for 99% of use cases. > the OS should work with hard disk manufacturers to make sure their devices meet whatever expectation the OS makes of them This is a bit of wishful thinking. We've seen already on numerous cases (Nvidia?) that OS developpers have very little leverage on hardware manufacturers. The best they could do is issue some kind of "Linux certified" label, which honestly nobody would care about. Also, I guess there is some form of optimum that users and providers reach in terms of stability versus performance. If one OS provides x10 performance for 99.99% of uses while an other one provides 100% safety for /10 performance, I'm not sure users would choose the latter.

josephg 3y ago
> Not to mention, that would basically disallow any kind of OS or hardware caching, which is immensely important for 99% of use cases. Why would it disallow caches? Just issue writes to a new location on disk and switch reads across to it after the write transaction has committed. In any case, most writes aren’t performance sensitive. I’m thinking things like saving a file when code editing. My editor shouldn’t need to do some obscure, fragile renaming dance just to overwrite a file safely. It wouldn’t write 10x slower. Just have the OS do what it needs to do so my editor doesn’t have to. And big transactions (like installing software or OS updates) could just do the whole update to a new location and then atomically swap to it. It shouldn’t cost any performance at all beyond a few pointers being moved around at the end of the transaction. Maybe doing it for all writes is too much. Let databases opt out if they want to do their own thing. But I think it would be a much better default. Right now all good software does some version of this themselves. An OS level solution would allow all those hacks to be removed. If anything, I’d expect performance to improve.
- Galanwe 3y ago
  I think we have a quiproquo about the level of atomicity here. You seem to be talking about _application level_ atomicity, like when you issue 4 write() to a file and want an "all or nothing" flush of these 4 writes. I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk". Am I understanding correctly?
  - josephg 3y ago
    Atomicity means something either happens entirely or it doesn't happen at all. > I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk". Thats not atomicity. Thats a blocking API. And we already have an fsync that does that. Blocking and atomicity are unrelated properties. You could have a blocking write thats not atomic (so if your computer turns off halfway through the write, it might have only half completed). And you can have an atomic write thats non-blocking (async). Or both or neither.
    - Galanwe 3y ago
      > Thats not atomicity. Thats a blocking API. > Blocking and atomicity are unrelated It's the same in this context. fsync uses a blocking API to implement atomicity. That is, after an fsync() all previous writes should have been written. > we already have an fsync that does that. Did you actually read the article...? The whole thing is about the fact that disk controllers can eagerly notify that a write was flushed to disk while in fact it is still in the disk write cache, and could have been either partially, fully or partly written, thus tricking fsync().
      - josephg 3y ago
        > It's the same in this context. fsync uses a blocking API to implement atomicity No, it doesn’t. In that context fsync is a write barrier. It doesn’t make writes atomic. The difference is this: If you issue a handful of writes then your computer crashes, some, all or none of the writes may have completed. And some writes might be “skewed” - where part of the write succeeded and part did not. Atomicity means a set of actions happens as one indivisible unit, and it either all happens or all doesn’t happen. Think database transactions. Filesystem writes are not like that. They are not atomic operations[1]. Fsync does not change the behaviour of the write system call. Fsync only stalls your program until all writes have been written to disk, at least assuming your drive isn’t lying to you. Just like a memory barrier. If your computer crashes before fsync returns, all bets are off. [1] With the exception of aligned writes under the device block size. Allegedly. I’d still like to see this actually documented somewhere.

josephg 3y ago

> Not to mention, that would basically disallow any kind of OS or hardware caching, which is immensely important for 99% of use cases. Why would it disallow caches? Just issue writes to a new location on disk and switch reads across to it after the write transaction has committed. In any case, most writes aren’t performance sensitive. I’m thinking things like saving a file when code editing. My editor shouldn’t need to do some obscure, fragile renaming dance just to overwrite a file safely. It wouldn’t write 10x slower. Just have the OS do what it needs to do so my editor doesn’t have to. And big transactions (like installing software or OS updates) could just do the whole update to a new location and then atomically swap to it. It shouldn’t cost any performance at all beyond a few pointers being moved around at the end of the transaction. Maybe doing it for all writes is too much. Let databases opt out if they want to do their own thing. But I think it would be a much better default. Right now all good software does some version of this themselves. An OS level solution would allow all those hacks to be removed. If anything, I’d expect performance to improve.

Galanwe 3y ago
I think we have a quiproquo about the level of atomicity here. You seem to be talking about _application level_ atomicity, like when you issue 4 write() to a file and want an "all or nothing" flush of these 4 writes. I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk". Am I understanding correctly?
- josephg 3y ago
  Atomicity means something either happens entirely or it doesn't happen at all. > I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk". Thats not atomicity. Thats a blocking API. And we already have an fsync that does that. Blocking and atomicity are unrelated properties. You could have a blocking write thats not atomic (so if your computer turns off halfway through the write, it might have only half completed). And you can have an atomic write thats non-blocking (async). Or both or neither.
  - Galanwe 3y ago
    > Thats not atomicity. Thats a blocking API. > Blocking and atomicity are unrelated It's the same in this context. fsync uses a blocking API to implement atomicity. That is, after an fsync() all previous writes should have been written. > we already have an fsync that does that. Did you actually read the article...? The whole thing is about the fact that disk controllers can eagerly notify that a write was flushed to disk while in fact it is still in the disk write cache, and could have been either partially, fully or partly written, thus tricking fsync().
    - josephg 3y ago
      > It's the same in this context. fsync uses a blocking API to implement atomicity No, it doesn’t. In that context fsync is a write barrier. It doesn’t make writes atomic. The difference is this: If you issue a handful of writes then your computer crashes, some, all or none of the writes may have completed. And some writes might be “skewed” - where part of the write succeeded and part did not. Atomicity means a set of actions happens as one indivisible unit, and it either all happens or all doesn’t happen. Think database transactions. Filesystem writes are not like that. They are not atomic operations[1]. Fsync does not change the behaviour of the write system call. Fsync only stalls your program until all writes have been written to disk, at least assuming your drive isn’t lying to you. Just like a memory barrier. If your computer crashes before fsync returns, all bets are off. [1] With the exception of aligned writes under the device block size. Allegedly. I’d still like to see this actually documented somewhere.