3 ms·
Disks, both spinning and SSD, lie about data durability more often then you would think. This is particularly a problem with consumer disks and write-back cache
by programd 7y ago
Disks, both spinning and SSD, lie about data durability more often then you would think. This is particularly a problem with consumer disks and write-back caches. Postgres has a good state of play writeup on reliably writing data to storage in their "Reliability and the Write-Ahead Log" documentation [1].
My rule of thumb is that if I want serious reliability I make sure my data winds up on three different pieces of hardware and includes CRC codes. So, basically ZFS or some cloud/cluster equivalent like S3.
On the other hand, for day to day work on your dev machine, the drives are so crazy reliable that you just don't worry about it. Which, of course, can bite you if you translate your day to day experience to production at scale. Everything breaks at scale.
[1] https://www.postgresql.org/docs/devel/wal-reliability.html https://www.postgresql.org/docs/devel/wal-reliability.html
- ecnahc515 7y agoWe need a "falsehoods programmers believe about disk reliability".
- erik_seaberg 7y ago> three different pieces of hardware This. Data is real when it has been acked by a quorum with independent power. Three is because two doesn't give you enough slack to maintain it.