4 ms·

> How can software possibly be expected to ensure what a physical device returns after the system restarts? By giving instructions to the device that the devic

> How can software possibly be expected to ensure what a physical device returns after the system restarts? By giving instructions to the device that the device understands and provides guarantees for...

chrisseaton 6y ago
How can the device provide those guarantees? If the system crashes hard in a power surge and the head crashes there's not going to be any use in claiming 'but the API guaranteed it'. There's no true durability.
- mehrdadn 6y ago
  > How can the device provide those guarantees? If the system crashes hard in a power surge and the head crashes there's not going to be any use in claiming 'but the API guaranteed it'. There's no true durability. By your logic there is no such thing as a guarantee in this world, period. A meteor could crash into Earth and wreck everything.
  - chrisseaton 6y ago
    > Very astute observation. You're being snarky. That's not allowed in the rules here and it's unkind even if it wasn't. > By your logic there is no such thing as a guarantee in this world, period. So what does the guarantee really mean? That the data will be on disk... as long as the crash isn't too hard? What do you usefully do with this guarantee? How is it better than no guarantee?
    - mehrdadn 6y ago
      Okay, I removed that phrase, but what do you want me to say here? Entire disciplines have devoted decades of research into providing useful guarantees and yet you're here observing that no human could possibly prevent some disasters... and using that logic to just trash everything and everyone's associated work along the way. I mean, you're clearly technically right that there is no such thing as an absolute guarantee, but... what am I supposed to do with this assertion? Find it insightful? Give up on the notion of a guarantee? Declare everything is useless? What response were you hoping for that you're still not getting?
      - chrisseaton 6y ago
        Why do you think the phrasing 'shall request ... is intended to' means there? It's a trapdoor for 'actually guarantees nothing', right?
        mehrdadn 6y ago
        It means: "In case we didn't get the point across in the technical language above, this is what we're trying to say in plain English." It does not mean "we're just writing this superfluous text to clock in extra hours or pass a word count threshold".
        chrisseaton 6y ago
        > It does not mean "we're just writing this superfluous text to clock in extra hours or pass a word count threshold". Where have I said I thought it was about writing time or a word count? You've imagined that.
        mehrdadn 6y ago
        Well I can't guess what else you imagined it to mean besides those, but I hope it's clarified now.
        chrisseaton 6y ago
        Not really - the language still seems super vague - intentionally so and thus useless. But thanks for trying despite your for some reason bitter snark and sarcasm!
        mehrdadn 6y ago
        I'm still lost what kind of a response you were hoping for that you didn't already receive, but you're welcome.
    - icedchai 6y ago
      I take it to mean that absent some unexpected, external event (like a head crash, flood, meteor, EMP, etc...) data will be on disk.
- tzs 6y ago
  You make it a requirement for some certification that the device maker wants to get. I once wrote the Novell NetWare drivers for a SCSI host adapter vendor, and part of Novell’s certification testing was to run I/O heavy tests on a system that would randomly cut power to the drive or the computer and make sure that this did not corrupt things beyond what NetWare could fully recover from. Hardware vendors like host adapter makers and drive makers wanted Novell certification, so made sure that was possible. They might also provide settings that sacrificed that for more performance, but they would make sure you could get decent performance using the full durability settings.
- Dylan16807 6y ago
  The device would guarantee that the data is safe as long as it doesn't get physically damaged. That's an extremely useful guarantee. Power outages are common and almost never cause physical damage. A flash device that isn't designed carefully enough could lose multiple megabytes of semi-random sectors if it gets interrupted at the wrong time, as a consequence of giant erase blocks.
formerly_proven 6y ago
On hard drives, even SATA ones this is mostly sorted out since a few years (Linux doesn't use FUA over SATA, leaves write-back drive cache enabled, and just issues a full cache flush, which the big majority of drives respect, more or less). SSDs, less so, fsync is a rather relative affair on some of them. IMHO, from having spent a fair amount of time on this, durability is a "fluid" concept, especially on desktop hardware (Microsoft had the right idea when they built Win Update on TxF), so if the data is important enough to worry about it, you should architect the system in a way that the durability of a fildes or drive is irrelevant to the overall system.
- mehrdadn 6y ago
  Right, but that's only because drives are lying at that point (or implemented so incompetently they have no idea they're lying...). Linux is trying to force liars to do what it wants anyway, and hence as a practical matter, programs need to worry about this too. This doesn't mean the spec is somehow broken or that honest (and competent) implementers couldn't guarantee durability if they wanted to.

chrisseaton 6y ago

How can the device provide those guarantees? If the system crashes hard in a power surge and the head crashes there's not going to be any use in claiming 'but the API guaranteed it'. There's no true durability.

mehrdadn 6y ago
> How can the device provide those guarantees? If the system crashes hard in a power surge and the head crashes there's not going to be any use in claiming 'but the API guaranteed it'. There's no true durability. By your logic there is no such thing as a guarantee in this world, period. A meteor could crash into Earth and wreck everything.
- chrisseaton 6y ago
  > Very astute observation. You're being snarky. That's not allowed in the rules here and it's unkind even if it wasn't. > By your logic there is no such thing as a guarantee in this world, period. So what does the guarantee really mean? That the data will be on disk... as long as the crash isn't too hard? What do you usefully do with this guarantee? How is it better than no guarantee?
  - mehrdadn 6y ago
    Okay, I removed that phrase, but what do you want me to say here? Entire disciplines have devoted decades of research into providing useful guarantees and yet you're here observing that no human could possibly prevent some disasters... and using that logic to just trash everything and everyone's associated work along the way. I mean, you're clearly technically right that there is no such thing as an absolute guarantee, but... what am I supposed to do with this assertion? Find it insightful? Give up on the notion of a guarantee? Declare everything is useless? What response were you hoping for that you're still not getting?
    - chrisseaton 6y ago
      Why do you think the phrasing 'shall request ... is intended to' means there? It's a trapdoor for 'actually guarantees nothing', right?
      - mehrdadn 6y ago
        It means: "In case we didn't get the point across in the technical language above, this is what we're trying to say in plain English." It does not mean "we're just writing this superfluous text to clock in extra hours or pass a word count threshold".
        chrisseaton 6y ago
        > It does not mean "we're just writing this superfluous text to clock in extra hours or pass a word count threshold". Where have I said I thought it was about writing time or a word count? You've imagined that.
        mehrdadn 6y ago
        Well I can't guess what else you imagined it to mean besides those, but I hope it's clarified now.
        chrisseaton 6y ago
        Not really - the language still seems super vague - intentionally so and thus useless. But thanks for trying despite your for some reason bitter snark and sarcasm!
        mehrdadn 6y ago
        I'm still lost what kind of a response you were hoping for that you didn't already receive, but you're welcome.
  - icedchai 6y ago
    I take it to mean that absent some unexpected, external event (like a head crash, flood, meteor, EMP, etc...) data will be on disk.
tzs 6y ago
You make it a requirement for some certification that the device maker wants to get. I once wrote the Novell NetWare drivers for a SCSI host adapter vendor, and part of Novell’s certification testing was to run I/O heavy tests on a system that would randomly cut power to the drive or the computer and make sure that this did not corrupt things beyond what NetWare could fully recover from. Hardware vendors like host adapter makers and drive makers wanted Novell certification, so made sure that was possible. They might also provide settings that sacrificed that for more performance, but they would make sure you could get decent performance using the full durability settings.
Dylan16807 6y ago
The device would guarantee that the data is safe as long as it doesn't get physically damaged. That's an extremely useful guarantee. Power outages are common and almost never cause physical damage. A flash device that isn't designed carefully enough could lose multiple megabytes of semi-random sectors if it gets interrupted at the wrong time, as a consequence of giant erase blocks.

formerly_proven 6y ago

On hard drives, even SATA ones this is mostly sorted out since a few years (Linux doesn't use FUA over SATA, leaves write-back drive cache enabled, and just issues a full cache flush, which the big majority of drives respect, more or less). SSDs, less so, fsync is a rather relative affair on some of them. IMHO, from having spent a fair amount of time on this, durability is a "fluid" concept, especially on desktop hardware (Microsoft had the right idea when they built Win Update on TxF), so if the data is important enough to worry about it, you should architect the system in a way that the durability of a fildes or drive is irrelevant to the overall system.

mehrdadn 6y ago
Right, but that's only because drives are lying at that point (or implemented so incompetently they have no idea they're lying...). Linux is trying to force liars to do what it wants anyway, and hence as a practical matter, programs need to worry about this too. This doesn't mean the spec is somehow broken or that honest (and competent) implementers couldn't guarantee durability if they wanted to.