4 ms·

Fact of the matter is, when a disk suddenly loses power, all bets are off. You can design intricate and complex systems of buffers all you want, but when the ch

Fact of the matter is, when a disk suddenly loses power, all bets are off. You can design intricate and complex systems of buffers all you want, but when the chips are down, at some point bits need to be flipped in the right positions to store the data, and for this you need electricity. The only way around this, would be if disks incorporated some kind of internal backup power system, which kicks in and flushes its internal buffers in case of a power loss. However, even that doesn't really solve the problem, it just moves it somewhere else...because the internal backup power is just another system that can fail.

masklinn 5y ago
> Fact of the matter is, when a disk suddenly loses power, all bets are off. The problem is not “the disk lost power during a write”, the problem is “I performed a write, the disk acknowledged the write, some time later it lost power”. > The only way around this, would be if disks incorporated some kind of internal backup power system, which kicks in and flushes its internal buffers in case of a power loss. That actually happens, caps can be used thus. Apparently an alternative (or complement?) for some SSDs is an SLC write cache (which is thus durable, not as fast as memory but much faster than TLC or somesuch).
- geocar 5y ago
  > The problem is not “the disk lost power during a write”, the problem is “I performed a write, the disk acknowledged the write, some time later it lost power”. I don't think this is right. There are a lot of things that can happen after the "disk" acknowledged the write that are a lot more common than a power failure (especially in a machine with batteries!) that fsync() does nothing about, so a system that gets its reliability from fsync() cannot in practice offer any more data persistence guarantees than one that doesn't.
  - masklinn 5y ago
    > I don't think this is right. You can think what you want, the entire thing started from marcan (re)discovering that fsync() on macos only flushes from FS to disk but does not require any guarantee of the disk. > There are a lot of things that can happen after the "disk" acknowledged the write that are a lot more common than a power failure Like what, putting a bullet through the SSD? > especially in a machine with batteries! Not every machine has a battery. Not every SSD is internal and hooked onto that battery. > a system that gets its reliability from fsync() cannot in practice offer any more data persistence guarantees than one that doesn't. So your point of view is what, reliability nihilism where data does not exist and death comes to us all?
- cout 5y ago
  Both are problems. In one case you care about consistency: how can you guarantee a transaction fully succeeded vs only partially succeeded if the drive tells you the data was written, even though it wasn't? In the other case, e.g. writing a log file or saving your word processing document, you want to ensure as much data is written as possible, even if out of order, so you can recover at least some of your work in the event of a failure.
  - masklinn 5y ago
    > Both are problems. One is what the discussion is about and the other is a completely different discussion nobody is actually having. And I don’t think many people actually care about what happens when you just yolo your writes, it’s not like it’s going to be anything you can usefully rely on.

masklinn 5y ago

> Fact of the matter is, when a disk suddenly loses power, all bets are off. The problem is not “the disk lost power during a write”, the problem is “I performed a write, the disk acknowledged the write, some time later it lost power”. > The only way around this, would be if disks incorporated some kind of internal backup power system, which kicks in and flushes its internal buffers in case of a power loss. That actually happens, caps can be used thus. Apparently an alternative (or complement?) for some SSDs is an SLC write cache (which is thus durable, not as fast as memory but much faster than TLC or somesuch).

geocar 5y ago
> The problem is not “the disk lost power during a write”, the problem is “I performed a write, the disk acknowledged the write, some time later it lost power”. I don't think this is right. There are a lot of things that can happen after the "disk" acknowledged the write that are a lot more common than a power failure (especially in a machine with batteries!) that fsync() does nothing about, so a system that gets its reliability from fsync() cannot in practice offer any more data persistence guarantees than one that doesn't.
- masklinn 5y ago
  > I don't think this is right. You can think what you want, the entire thing started from marcan (re)discovering that fsync() on macos only flushes from FS to disk but does not require any guarantee of the disk. > There are a lot of things that can happen after the "disk" acknowledged the write that are a lot more common than a power failure Like what, putting a bullet through the SSD? > especially in a machine with batteries! Not every machine has a battery. Not every SSD is internal and hooked onto that battery. > a system that gets its reliability from fsync() cannot in practice offer any more data persistence guarantees than one that doesn't. So your point of view is what, reliability nihilism where data does not exist and death comes to us all?
cout 5y ago
Both are problems. In one case you care about consistency: how can you guarantee a transaction fully succeeded vs only partially succeeded if the drive tells you the data was written, even though it wasn't? In the other case, e.g. writing a log file or saving your word processing document, you want to ensure as much data is written as possible, even if out of order, so you can recover at least some of your work in the event of a failure.
- masklinn 5y ago
  > Both are problems. One is what the discussion is about and the other is a completely different discussion nobody is actually having. And I don’t think many people actually care about what happens when you just yolo your writes, it’s not like it’s going to be anything you can usefully rely on.