3 ms·

Please, where can i read more about this. I remember bricking OCZ drive by setting ATA password, as was fashionable to do back then, but 1% writes going to wron

by 112233 3y ago

Please, where can i read more about this. I remember bricking OCZ drive by setting ATA password, as was fashionable to do back then, but 1% writes going to wrong sector - what are these drives, fake sd cards from aliexpress? Like, which manufacturer goes, like "tests show we cannot write more that 400kB without corrupting drive, let us ship this!" ?

londons_explore 3y ago
Writes going to the wrong sector are usually wear levelling algorithms gone wrong. Specifically, it normally means the information about which logical sector maps to which physical sector was updated not in sync with the actual writing of the data. This is a common performance 'trick' - by delaying and aggregating these bookkeeping writes, and taking them off the critical path, you avoid writing so much data and the user sees lower latency. However, if something like a power failure or firmware crash happens, and the bookkeeping writes never happen, then the end result that the user sees after a reboot is their data written to the wrong sector.
jlokier 3y ago
> Like, which manufacturer goes, like "tests show we cannot write more that 400kB without corrupting drive, let us ship this!" ? According to https://www.sqlite.org/howtocorrupt.html https://www.sqlite.org/howtocorrupt.html there are such drives: 4.2. Fake capacity USB sticks There are many fraudulent USB sticks in circulation that report to have a high capacity (ex: 8GB) but are really only capable of storing a much smaller amount (ex: 1GB). Attempts to write on these devices will often result in unrelated files being overwritten. Any use of a fraudulent flash memory device can easily lead to database corruption, therefore. Internet searches such as "fake capacity usb" will turn up lots of disturbing information about this problem. Bit flips amd overwriting wrong sectors from unrelated files being written are also mentioned. You might think this sort of thing is just cheap USB flash drives, bit I've been told about NVMe SSDs violating their guarantees and causing very strange corruption patterns too. Unfortunately when the cause is a bug in the storage's device own algorithms, the rare corruption event is not necessarily limited to a few bytes or just 1 sector here or there, nor to just the sectors being written. I don't know how prevalent any of these things are really. The sqlite.org says "most" consumer HDDs lie about commiting data to the platter before reporting they've done so, but when I worked with ext3 barriers back in the mid 2000s, the HDDs I tested had timing consistent with flushing write cache correctly, and turning off barriers did in fact lead to observable filesystem corruption on power loss, which was prevented by turning on barriers. The barriers were so important they made the difference between embedded devices that could be reliably power cycled, vs those which didn't reliably recover on boot.
- Dylan16807 3y ago
  > Fake capacity USB sticks Those drives have a sudden flip from 0% corruption to 95-100% corruption when you hit their limits. I wouldn't count that as the same thing. And you can't reasonably expect anything to work on those. > The sqlite.org says "most" consumer HDDs lie about commiting data to the platter before reporting they've done so, but when I worked with ext3 barriers back in the mid 2000s, the HDDs I tested had timing consistent with flushing write cache correctly Losing a burst of writes every once in a while also manifests extremely differently from steady 1% loss and needs to be handled in a very different way. And if it's at power loss it might be as simple as rolling back the last checkpoint during mount if verification fails.

by 112233 3y ago

londons_explore 3y ago

Writes going to the wrong sector are usually wear levelling algorithms gone wrong. Specifically, it normally means the information about which logical sector maps to which physical sector was updated not in sync with the actual writing of the data. This is a common performance 'trick' - by delaying and aggregating these bookkeeping writes, and taking them off the critical path, you avoid writing so much data and the user sees lower latency. However, if something like a power failure or firmware crash happens, and the bookkeeping writes never happen, then the end result that the user sees after a reboot is their data written to the wrong sector.

jlokier 3y ago

> Like, which manufacturer goes, like "tests show we cannot write more that 400kB without corrupting drive, let us ship this!" ? According to https://www.sqlite.org/howtocorrupt.html https://www.sqlite.org/howtocorrupt.html there are such drives: 4.2. Fake capacity USB sticks There are many fraudulent USB sticks in circulation that report to have a high capacity (ex: 8GB) but are really only capable of storing a much smaller amount (ex: 1GB). Attempts to write on these devices will often result in unrelated files being overwritten. Any use of a fraudulent flash memory device can easily lead to database corruption, therefore. Internet searches such as "fake capacity usb" will turn up lots of disturbing information about this problem. Bit flips amd overwriting wrong sectors from unrelated files being written are also mentioned. You might think this sort of thing is just cheap USB flash drives, bit I've been told about NVMe SSDs violating their guarantees and causing very strange corruption patterns too. Unfortunately when the cause is a bug in the storage's device own algorithms, the rare corruption event is not necessarily limited to a few bytes or just 1 sector here or there, nor to just the sectors being written. I don't know how prevalent any of these things are really. The sqlite.org says "most" consumer HDDs lie about commiting data to the platter before reporting they've done so, but when I worked with ext3 barriers back in the mid 2000s, the HDDs I tested had timing consistent with flushing write cache correctly, and turning off barriers did in fact lead to observable filesystem corruption on power loss, which was prevented by turning on barriers. The barriers were so important they made the difference between embedded devices that could be reliably power cycled, vs those which didn't reliably recover on boot.

Dylan16807 3y ago
> Fake capacity USB sticks Those drives have a sudden flip from 0% corruption to 95-100% corruption when you hit their limits. I wouldn't count that as the same thing. And you can't reasonably expect anything to work on those. > The sqlite.org says "most" consumer HDDs lie about commiting data to the platter before reporting they've done so, but when I worked with ext3 barriers back in the mid 2000s, the HDDs I tested had timing consistent with flushing write cache correctly Losing a burst of writes every once in a while also manifests extremely differently from steady 1% loss and needs to be handled in a very different way. And if it's at power loss it might be as simple as rolling back the last checkpoint during mount if verification fails.