5 ms·

That's not something you can do reliably in software, datacenter grade NVMe drives come with power loss protection and additional capacitors to handle that grac

That's not something you can do reliably in software, datacenter grade NVMe drives come with power loss protection and additional capacitors to handle that gracefully. If power is cut at the wrong moment the partition may not be mountable afterwards otherwise. If you really live somewhere with frequent outages, buy an industrial drive that has a PLP rating. Or get a UPS, they tend to be cheaper.

crote 10mo ago
Isn't that the entire point of write-ahead logs, journaling file systems, and fsync in general? A roll-back or roll-forward due to a power loss causing a partial write is completely expected, but surely consumer SSDs wouldn't just completely ignore fsync and blatantly lie that the data has been persisted? As I understood it, the capacitors on datacenter-grade drives are to give it more flexibility, as it allows the drive to issue a successful write response for cached data: the capacitor guarantees that even with a power loss the write will still finish, so for all intents and purposes it has been persisted, so an fsync can return without having to wait on the actual flash itself, which greatly increases performance. Have I just completely misunderstood this?
- Nextgrid 10mo ago
  > ignore fsync and blatantly lie that the data has been persisted Unfortunately they do: https://news.ycombinator.com/item?id=38371307 https://news.ycombinator.com/item?id=38371307
  - btown 10mo ago
    If the drives continue to have power, but the OS has crashed, will the drives persist the data once a certain amount of time has passed? Are datacenters set up to take advantage of this?
    - Nextgrid 10mo ago
      > will the drives persist the data once a certain amount of time has passed Yes, otherwise those drives wouldn't work at all and would have a 100% warranty return rate. The reason they get away with it is that the misbehavior is only a problem in a specific edge-case (forgetting data written shortly before a power loss).
    - unsnap_biceps 10mo ago
      Yes, the drives are unaware of the OS state.
- unsnap_biceps 10mo ago
  you actually don't need capacitors for rotating media, Western Digital has a feature called "ArmorCache" that uses the rotational energy in the platters to power the drive long enough to sync the volatile cache to a non volatile storage. https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/collateral/tech-brief/tech-brief-write-cache-data-safety.pdf https://documents.westerndigital.com/content/dam/doc-library...
  - toomuchtodo 10mo ago
    Very cool, like the ram air turbine that deploys on aircraft in the event of a power loss.
  - patmorgan23 10mo ago
    Good I love engineers
- Aerolfos 10mo ago
  > but surely consumer SSDs wouldn't just completely ignore fsync and blatantly lie that the data has been persisted? That doesn't even help if fsync() doesn't do what developers expect: https://danluu.com/fsyncgate/ https://danluu.com/fsyncgate/ I think this was the blog post that had a bunch more stuff that can go wrong too: https://danluu.com/deconstruct-files/ https://danluu.com/deconstruct-files/ But basically fsync itself (sometimes) has dubious behaviour, then OS on top of kernel handles it dubiously, and then even on top of that most databases can ignore fsync erroring (and lie that the data was written properly) So... yes.

crote 10mo ago

Isn't that the entire point of write-ahead logs, journaling file systems, and fsync in general? A roll-back or roll-forward due to a power loss causing a partial write is completely expected, but surely consumer SSDs wouldn't just completely ignore fsync and blatantly lie that the data has been persisted? As I understood it, the capacitors on datacenter-grade drives are to give it more flexibility, as it allows the drive to issue a successful write response for cached data: the capacitor guarantees that even with a power loss the write will still finish, so for all intents and purposes it has been persisted, so an fsync can return without having to wait on the actual flash itself, which greatly increases performance. Have I just completely misunderstood this?

Nextgrid 10mo ago
> ignore fsync and blatantly lie that the data has been persisted Unfortunately they do: https://news.ycombinator.com/item?id=38371307 https://news.ycombinator.com/item?id=38371307
- btown 10mo ago
  If the drives continue to have power, but the OS has crashed, will the drives persist the data once a certain amount of time has passed? Are datacenters set up to take advantage of this?
  - Nextgrid 10mo ago
    > will the drives persist the data once a certain amount of time has passed Yes, otherwise those drives wouldn't work at all and would have a 100% warranty return rate. The reason they get away with it is that the misbehavior is only a problem in a specific edge-case (forgetting data written shortly before a power loss).
  - unsnap_biceps 10mo ago
    Yes, the drives are unaware of the OS state.
unsnap_biceps 10mo ago
you actually don't need capacitors for rotating media, Western Digital has a feature called "ArmorCache" that uses the rotational energy in the platters to power the drive long enough to sync the volatile cache to a non volatile storage. https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/collateral/tech-brief/tech-brief-write-cache-data-safety.pdf https://documents.westerndigital.com/content/dam/doc-library...
- toomuchtodo 10mo ago
  Very cool, like the ram air turbine that deploys on aircraft in the event of a power loss.
- patmorgan23 10mo ago
  Good I love engineers
Aerolfos 10mo ago
> but surely consumer SSDs wouldn't just completely ignore fsync and blatantly lie that the data has been persisted? That doesn't even help if fsync() doesn't do what developers expect: https://danluu.com/fsyncgate/ https://danluu.com/fsyncgate/ I think this was the blog post that had a bunch more stuff that can go wrong too: https://danluu.com/deconstruct-files/ https://danluu.com/deconstruct-files/ But basically fsync itself (sometimes) has dubious behaviour, then OS on top of kernel handles it dubiously, and then even on top of that most databases can ignore fsync erroring (and lie that the data was written properly) So... yes.