5 ms·

One reason: If you send a flush command to a consumer grade hd, it returns before the flush is done. This makes it look nicer on older HD benchmarking software

One reason: If you send a flush command to a consumer grade hd, it returns before the flush is done. This makes it look nicer on older HD benchmarking software. Enterprise hds return when the flush is complete, so databases acknowledge a commit only when it was actually succesfull.

pkaye 6y ago
> If you send a flush command to a consumer grade hd, it returns before the flush is done. So essentially its a no-op?
- lukevp 6y ago
  Just speculating but perhaps the consumer ones return after the cache is populated and not after the data is durably on disk? That wouldn’t really be a no-op but would be much quicker for small writes at the expense of correctness in a power outage (although in the case of a DB the WAL entries are typically atomic so it would just be a rollback anyway).
- dijit 6y ago
  Think of it like mongodb thinks of data: it's eventually consistent, maybe.
polskibus 6y ago
How can I find out which drives exactly have synchronous flush and which don't? Is it a widely publicized fact?
Hello71 6y ago
source? this sounds like an urban myth. there's no way that every single consumer grade hard drive "returns before the flush is done", given that: 1. consumer grade is not well-defined, 2. there are many different "flush commands", and it does not depend on whether a drive is "consumer grade", and 3. if this were the case, then filesystem and database corruption would be rampant, which it is eminently not. it would be impossible to write a remotely reliable high-performance database or filesystem if flushing were useless. professionals doing formal analyses like https://www.sqlite.org/psow.html https://www.sqlite.org/psow.html would be a complete waste of time, because everyone would be just praying anyways.
- sandworm101 6y ago
  >> consumer grade is not well-defined WD has laid out a reasonable standard. Blue/Black/Green are consumer grades. Red is for NAS and Gold is for enterprise.
- kijiki 6y ago
  https://brad.livejournal.com/2116715.html https://brad.livejournal.com/2116715.html some enterprise SCSI drives too. This was all back in the late 90's/early 2000's, no idea if it still happens with SATA or SAS drives...
  - Hello71 6y ago
    sure, and "consumer" filesystems/databases were also pretty bad. ext3 was only released in 2001, and Windows supported FAT system disk since at least Vista. the "standard" OSS database was MySQL until at least 2010, if not later. I argue 2020 is different though. despite the onslaught of mongodb 'engineers', data reliability is far better than it was in 2000.
james412 6y ago
I think your wires may be a little crossed. This is unlikely to have ever been true for magnetic drives, but there is a quite significant grain of truth (no pun intended) to it for SSD. Consumer SSDs may indeed return an early indication that a flush is complete, but whether they are lying depends heavily on your notion of persistence: - the SSD may return done once the data is buffered in RAM - or once it has reached a temporary log - or once it has reached a 'permanent' location on flash - or once it has recorded the mapping of data<->location to some separate directory - or once it has updated some separate root pointer pointing to the latest version of the log/mapping structures in some separate special storage I don't believe any SSD waits until the last step above before returning done. Separately, the SSD: - may have no electrical power outage protection whatsoever - may have supercapacitors fitted - the supercap may guarantee full durability of every acknowledged flush - or only guarantee atomicity up to some prior acknowledged flush - or only guarantee that the drive has /some/ data on it in /some/ state on the next reboot SSDs are insanely complex
- kijiki 6y ago
  I'd been hearing rumors (possibly spread by the mfgs to scare enterprises away from cheap IDE drives) about it during the late 90's and early 2000's. Then bfitz proved in in 2005, and it turns out SCSI drives sometimes do it too: https://brad.livejournal.com/2116715.html https://brad.livejournal.com/2116715.html
- jrockway 6y ago
  It is interesting that to remain competitive in benchmarks, SSD vendors have to sell you an incredibly complicated software system alongside their flash chips soldered to a circuit board. They can't fix your filesystem, because you won't switch (and why pay them when your OS comes with one for free). They can't fix your database, because you won't switch (you already pay Amazon a million dollars a month for it). So they have to implement all the complicated parts of a database or filesystem, then try to understand the stream of events that real databases and real filesystems send it, and then restructure them to win the benchmark. All of this with no source code and probably no real test suite, and it all ends up being slower than a database that could just write to the flash chips directly. I am also "excited" when I think about how the SSD shards blocks across flash chips and adds erasure codes, then people put the SSDs in RAID to shard blocks across SSDs and add erasure codes, then people put a filesystem on top of that that shards blocks across arrays and adds erasure codes, then put an application on top of that that writes to 3 replicas and writes erasure codes to 2 more. Now your "hello world" text file uses 8 gigabytes of storage, but at least you only lose it if there's one bug in the SSD controller, your RAID software, your filesystem, or your application. Or a tornado hits the SSD. Oh well. HN often contains articles like "abstraction is bad", but even their contrived examples aren't this scary. I'm still pleasantly surprised when I log into my computer in the morning and still have a .bashrc.

pkaye 6y ago

> If you send a flush command to a consumer grade hd, it returns before the flush is done. So essentially its a no-op?

lukevp 6y ago
Just speculating but perhaps the consumer ones return after the cache is populated and not after the data is durably on disk? That wouldn’t really be a no-op but would be much quicker for small writes at the expense of correctness in a power outage (although in the case of a DB the WAL entries are typically atomic so it would just be a rollback anyway).
dijit 6y ago
Think of it like mongodb thinks of data: it's eventually consistent, maybe.

polskibus 6y ago

How can I find out which drives exactly have synchronous flush and which don't? Is it a widely publicized fact?

Hello71 6y ago

source? this sounds like an urban myth. there's no way that every single consumer grade hard drive "returns before the flush is done", given that: 1. consumer grade is not well-defined, 2. there are many different "flush commands", and it does not depend on whether a drive is "consumer grade", and 3. if this were the case, then filesystem and database corruption would be rampant, which it is eminently not. it would be impossible to write a remotely reliable high-performance database or filesystem if flushing were useless. professionals doing formal analyses like https://www.sqlite.org/psow.html https://www.sqlite.org/psow.html would be a complete waste of time, because everyone would be just praying anyways.

sandworm101 6y ago
>> consumer grade is not well-defined WD has laid out a reasonable standard. Blue/Black/Green are consumer grades. Red is for NAS and Gold is for enterprise.
kijiki 6y ago
https://brad.livejournal.com/2116715.html https://brad.livejournal.com/2116715.html some enterprise SCSI drives too. This was all back in the late 90's/early 2000's, no idea if it still happens with SATA or SAS drives...
- Hello71 6y ago
  sure, and "consumer" filesystems/databases were also pretty bad. ext3 was only released in 2001, and Windows supported FAT system disk since at least Vista. the "standard" OSS database was MySQL until at least 2010, if not later. I argue 2020 is different though. despite the onslaught of mongodb 'engineers', data reliability is far better than it was in 2000.

james412 6y ago

I think your wires may be a little crossed. This is unlikely to have ever been true for magnetic drives, but there is a quite significant grain of truth (no pun intended) to it for SSD. Consumer SSDs may indeed return an early indication that a flush is complete, but whether they are lying depends heavily on your notion of persistence: - the SSD may return done once the data is buffered in RAM - or once it has reached a temporary log - or once it has reached a 'permanent' location on flash - or once it has recorded the mapping of data<->location to some separate directory - or once it has updated some separate root pointer pointing to the latest version of the log/mapping structures in some separate special storage I don't believe any SSD waits until the last step above before returning done. Separately, the SSD: - may have no electrical power outage protection whatsoever - may have supercapacitors fitted - the supercap may guarantee full durability of every acknowledged flush - or only guarantee atomicity up to some prior acknowledged flush - or only guarantee that the drive has /some/ data on it in /some/ state on the next reboot SSDs are insanely complex

kijiki 6y ago
I'd been hearing rumors (possibly spread by the mfgs to scare enterprises away from cheap IDE drives) about it during the late 90's and early 2000's. Then bfitz proved in in 2005, and it turns out SCSI drives sometimes do it too: https://brad.livejournal.com/2116715.html https://brad.livejournal.com/2116715.html
jrockway 6y ago
It is interesting that to remain competitive in benchmarks, SSD vendors have to sell you an incredibly complicated software system alongside their flash chips soldered to a circuit board. They can't fix your filesystem, because you won't switch (and why pay them when your OS comes with one for free). They can't fix your database, because you won't switch (you already pay Amazon a million dollars a month for it). So they have to implement all the complicated parts of a database or filesystem, then try to understand the stream of events that real databases and real filesystems send it, and then restructure them to win the benchmark. All of this with no source code and probably no real test suite, and it all ends up being slower than a database that could just write to the flash chips directly. I am also "excited" when I think about how the SSD shards blocks across flash chips and adds erasure codes, then people put the SSDs in RAID to shard blocks across SSDs and add erasure codes, then people put a filesystem on top of that that shards blocks across arrays and adds erasure codes, then put an application on top of that that writes to 3 replicas and writes erasure codes to 2 more. Now your "hello world" text file uses 8 gigabytes of storage, but at least you only lose it if there's one bug in the SSD controller, your RAID software, your filesystem, or your application. Or a tornado hits the SSD. Oh well. HN often contains articles like "abstraction is bad", but even their contrived examples aren't this scary. I'm still pleasantly surprised when I log into my computer in the morning and still have a .bashrc.