5 ms·

Those who forget history are doomed to repeat it. Just seven years ago Crucial sold tens of thousands of their "M4" SSDs with a firmware bug that made them fai

Those who forget history are doomed to repeat it. Just seven years ago Crucial sold tens of thousands of their "M4" SSDs with a firmware bug that made them fail after 5184 hours: https://www.anandtech.com/show/5424/crucial-provides-a-firmware-update-for-m4-to-fix-the-bsod-issue https://www.anandtech.com/show/5424/crucial-provides-a-firmw... Do they still not test these things with artificially incremented counters?

djsmiley2k 7y ago
Some early Intel SSD's did the same thing, prior to M4's... haha
- jzwinck 7y ago
  Do you have a link for the old Intel bug? Here's one for a new Intel bug after just 1700 power-on hours on some enterprise-class SSDs that are still being sold today: https://www.intel.com/content/www/us/en/support/articles/000038720/memory-and-storage/data-center-ssds.html https://www.intel.com/content/www/us/en/support/articles/000... That's just 71 days of uptime and they hang. There are tens of thousands of these drives deployed as well.
  - Bootwizard 7y ago
    Why are they allowed to still sell these broken products? That's a scam as far as I'm concerned.
    - oarsinsync 7y ago
      They released a fix for existing devices, replaced affected devices that had been bricked, and included the fix as part of the manufacturing process for new devices being built. It was certainly very inconvenient having to reboot systems while we waited for a fix to exist, fortunately we didn't lose too many disks before the fault was identified, which reduced the man hours involved in the DCs. I'm not sure if any part of this is a scam. A bug, certainly.
  - rythie 7y ago
    New drives already had the updated firmware back in May when I bought some.
  - mtanski 7y ago
    Oh boy. We had somerhing like 5 our of 8 drives fail all at the same time. All of them were affected models bought at begining of summer and failed a couple months later. It was a pretty maddening thing to debug and figure out where the issue was (servers, rack, drives, RAID do controllers). 2 different machines 2 and 3 drives. Week later we found the Intel bulletin about the issue. Thank God for pgbackrest backups.
mhandley 7y ago
Boeing didn't even test their 787 aircraft for integer overflows, and that's in a safety-critical environment, so I'm not sure I'd expect SSD vendors to be any better. https://www.engadget.com/2015/05/01/boeing-787-dreamliner-software-bug/ https://www.engadget.com/2015/05/01/boeing-787-dreamliner-so... Not that throwing an exception on integer overflow is any better, unless you catch the exception. The classic example here is the Ariane 5 failure: http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html
- everybodyknows 7y ago
  The 787 case is most fascinating in that while the bug is dead simple, a fix is not. >Most importantly, the company's already working on an update that will patch the software vulnerability -- though there's no word on when its jets will receive it. My search of DDG turned up nothing about a resolution. Anyone know? I know what I would recommend, but marketing would not like it ;-)
- planteen 7y ago
  This same problem also led to the loss of the Deep Impact spacecraft on its extended mission: "On September 20, 2013, NASA abandoned further attempts to contact the craft.[77] According to chief scientist A'Hearn,[78] the reason for the software malfunction was a Y2K-like problem. August 11, 2013, 00:38:49, was 232 tenth-seconds from January 1, 2000, leading to speculation that a system on the craft tracked time in one-tenth second increments since January 1, 2000, and stored it in an unsigned 32-bit integer, which then overflowed at this time, similar to the Year 2038 problem" https://en.wikipedia.org/wiki/Deep_Impact_(spacecraft)#Contact_lost_and_end_of_mission https://en.wikipedia.org/wiki/Deep_Impact_(spacecraft)#Conta...
  - Tempest1981 7y ago
    Your superscript got eaten. 2^32
andonisus 7y ago
This exact thing happened to me. I went crazy for a week straight testing every other component of my PC. I was convinced it was the graphics cards drawing too much power. Then it was clear that my OS was corrupted and needed to be reinstalled. Finally, I found an obscure forum post telling me about a firmware bug happening at ~5K hours of disk usage. I updated the firmware and haven't had an issue since.
touisteur 7y ago
I thought the coding/design pattern was to set the initial value of any counter 1 minute (or whatever eon makes sense in your application) from the roll-over so you'd see it 'right away' if it was badly handled. It's like you should use specific types with default values...

djsmiley2k 7y ago

Some early Intel SSD's did the same thing, prior to M4's... haha

jzwinck 7y ago
Do you have a link for the old Intel bug? Here's one for a new Intel bug after just 1700 power-on hours on some enterprise-class SSDs that are still being sold today: https://www.intel.com/content/www/us/en/support/articles/000038720/memory-and-storage/data-center-ssds.html https://www.intel.com/content/www/us/en/support/articles/000... That's just 71 days of uptime and they hang. There are tens of thousands of these drives deployed as well.
- Bootwizard 7y ago
  Why are they allowed to still sell these broken products? That's a scam as far as I'm concerned.
  - oarsinsync 7y ago
    They released a fix for existing devices, replaced affected devices that had been bricked, and included the fix as part of the manufacturing process for new devices being built. It was certainly very inconvenient having to reboot systems while we waited for a fix to exist, fortunately we didn't lose too many disks before the fault was identified, which reduced the man hours involved in the DCs. I'm not sure if any part of this is a scam. A bug, certainly.
- rythie 7y ago
  New drives already had the updated firmware back in May when I bought some.
- mtanski 7y ago
  Oh boy. We had somerhing like 5 our of 8 drives fail all at the same time. All of them were affected models bought at begining of summer and failed a couple months later. It was a pretty maddening thing to debug and figure out where the issue was (servers, rack, drives, RAID do controllers). 2 different machines 2 and 3 drives. Week later we found the Intel bulletin about the issue. Thank God for pgbackrest backups.

mhandley 7y ago

Boeing didn't even test their 787 aircraft for integer overflows, and that's in a safety-critical environment, so I'm not sure I'd expect SSD vendors to be any better. https://www.engadget.com/2015/05/01/boeing-787-dreamliner-software-bug/ https://www.engadget.com/2015/05/01/boeing-787-dreamliner-so... Not that throwing an exception on integer overflow is any better, unless you catch the exception. The classic example here is the Ariane 5 failure: http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html

everybodyknows 7y ago
The 787 case is most fascinating in that while the bug is dead simple, a fix is not. >Most importantly, the company's already working on an update that will patch the software vulnerability -- though there's no word on when its jets will receive it. My search of DDG turned up nothing about a resolution. Anyone know? I know what I would recommend, but marketing would not like it ;-)
planteen 7y ago
This same problem also led to the loss of the Deep Impact spacecraft on its extended mission: "On September 20, 2013, NASA abandoned further attempts to contact the craft.[77] According to chief scientist A'Hearn,[78] the reason for the software malfunction was a Y2K-like problem. August 11, 2013, 00:38:49, was 232 tenth-seconds from January 1, 2000, leading to speculation that a system on the craft tracked time in one-tenth second increments since January 1, 2000, and stored it in an unsigned 32-bit integer, which then overflowed at this time, similar to the Year 2038 problem" https://en.wikipedia.org/wiki/Deep_Impact_(spacecraft)#Contact_lost_and_end_of_mission https://en.wikipedia.org/wiki/Deep_Impact_(spacecraft)#Conta...
- Tempest1981 7y ago
  Your superscript got eaten. 2^32

andonisus 7y ago

This exact thing happened to me. I went crazy for a week straight testing every other component of my PC. I was convinced it was the graphics cards drawing too much power. Then it was clear that my OS was corrupted and needed to be reinstalled. Finally, I found an obscure forum post telling me about a firmware bug happening at ~5K hours of disk usage. I updated the firmware and haven't had an issue since.

touisteur 7y ago

I thought the coding/design pattern was to set the initial value of any counter 1 minute (or whatever eon makes sense in your application) from the roll-over so you'd see it 'right away' if it was badly handled. It's like you should use specific types with default values...