5 ms·

One thing that HDDs seem to have and SSDs seem to lack is that they fail _reliably_. Some SMART values get higher, warning, then error. Sometimes directly in er

One thing that HDDs seem to have and SSDs seem to lack is that they fail _reliably_. Some SMART values get higher, warning, then error. Sometimes directly in error. Tick tick tick tick. But you often can still recover some data. SSDs on the other hand: from one day to the next they are dead, don't do NVME or SATA handshakes anymore, and you're done. I wonder if SSD firmware still needs to mature to the point of still allowing some connectivity and data recovery.

hammyhavoc 3y ago
Yes, I have suffered data loss from both HDDs and SSDs, paid for the recovery services available, and HDDs have always been the ones to see any degree of success. SSDs? You're probably screwed.
- quags 3y ago
  I’ve done three recoveries to a remote service for clients - the first failed and two which were ssd had a full success.m but all were ssds. Hdds I never sent to recovery because generally they were not complete failures (ie some bad blocks and a copy around was possible ) or the board could be replaced to fix it with the same model and firmware.
- MichaelZuo 3y ago
  Meaningful data recovery from a failed SSD could need tens of millions of dollars worth of equipment, depending on how it failed.
  - CyberDildonics 3y ago
    Can you give a source for why you think that?
hilbert42 3y ago
"I wonder if SSD firmware still needs to mature to the point..." This is a major issue, I've had almost brand new SSDs fail catastrophically without warning. In one instance I'd gone to great trouble to create an organized data disk for archiving and it failed just as I was about to back it up, and nothing I did could get it to even recognize the SATA interface let alone give me access to the data. This is most annoying, first, why does the interface have to also die when something goes wrong with the memory? Second, can't the firmware at least tell us what's wrong with the memory? (Given the interface is essentially the smallest part of the drive's real estate it's unlikely the bit that fails, similarly the firmware would be second smallest—even then, why isn't redundancy built into drives for these two vital components (two of each)?) So what's going on? To maintain the integrity of our data we not only need these issues solved but also we need to know much more about the architecture of SSD drives. As users, we ought to pay much more attention to the fact that much of this problem stems from the inordinate amount of secrecy SSD manufacturers adopt. We really do need some new manufacturer to come to market with a drive based on data integrity, its selling point would be 'it's open, here's what we do and here are our tools to help you recover your data'. There's also the longevity problem that needs resolution, I've put SSD as backups into storage only to come back several years later to find the data missing. As it is, the data on HDDs is ephemeral enough, but relying on a 'well' of electrons trapped in 'glass' is not only risky but almost foolhardy. If it had not been for the speed and convenience of SSDs we'd never have contemplated using them in the first instance. Before we scrap HDDs we really do need much more reliable data storage than SSDs, especially so for archival data. It remains to be seen what eventuates, but whatever it is it should be based on a technology (physics) that's inherently reliable and stable, crystals for instance.
- wtallis 3y ago
  > This is most annoying, first, why does the interface have to also die when something goes wrong with the memory? I think you've read too many horror stories about unreliability of flash memory and are now blaming the flash memory for everything that goes wrong with an SSD even in instances that look very likely to be a firmware bug rather than a direct result of flash memory failing. Flash memory is error-prone, but largely in predictable ways. SSD firmware is a pile of overcomplicated software developed in a hurry and squeezed into an embedded system with limited resources. Do you really think the former is more likely to randomly cause catastrophic failures? > Given the interface is essentially the smallest part of the drive's real estate it's unlikely the bit that fails, similarly the firmware would be second smallest—even then, why isn't redundancy built into drives for these two vital components (two of each)? Maybe you're judging "smallest" in terms of transistor count or something, but your metric should be complexity. A NAND flash memory chip is a massive array of highly-regular memory cells that have been thoroughly characterized both in terms of aggregate/average behavior and localized defect rates. By contrast, SSD firmware is among other things implementing at least half the functionality of a filesystem, and the host interface of the SSD has to interact with third-party components that only loosely comply with protocol specifications that are themselves sometimes shockingly vague or silent on important questions.
  - hilbert42 3y ago
    "think you've read too many horror stories about unreliability of flash memory..." Given my collection of dozens of dead SSDs I have my own horror stories, my comments are based on painful experience. In most instances whenever I've had a HDD fail—and that's many over the years—I have been able to recover the data but I cannot say that about SSDs or USB thumb drives as they mostly fail catastrophically and without warning—and the data is irrecoverable. Moreover, with SSDs, in cases where data has been accidentally erased, wear leveling often mitigates against successful recovery. I've lost far more data to SSDs than I ever have to HDDs yet I've only been using them for about one third as long. I've now reached the point where I don't bother to recover data from a dead SSD as I know it'll likely be a futile effort. That's still not so with HDDs but there are exceptions. Moreover, whether it's hardware failure or accidental data erasure, ALL SSD and USB thumb drive manufacturers either provide no recovery tools or if they do then they are inadequate almost to the point of being useless (for instance, SanDisk's recovery tool is hopeless, I've just unsuccessfully tried it on its own 256GB SD cards). My contention is that if the firmware architectures of these devices were actually designed to be aware of file structures then recovery would be easier. Essentially, they're still not user-friendly and manufacturers have made little or no effort to rectify the problems. "SSD firmware is a pile of overcomplicated software developed in a hurry and squeezed into an embedded system with limited resources." Limited resources, why? A 500GB SSD is orders of magnitude larger than even the most bloated firmware would occupy. (The world' most bloated software Windows only takes about 7GB!) If SSD firmware is developed in such a hurry and thus jeopardizes users' data then I would contend the products are not fit for purpose. The fact remains that SSDs are intentionally made as 'black boxes' by their manufacturers and I'd maintain that this is highly unsatisfactory as far as users are concerned. We ought to be long past the point where users' data are put at risk by shoddy or limited design efforts and or the desire of manufacturers to keep their designs secret. Nowadays there's too much at stake just to hold the world's data inside black boxes that no one understands—ones that not even manufacturers can recover (or are willing to recover) data from when they fail. We users deserve much better. [In saying that I'm no apologist for HDD manufacturers who adopt similar tactics.] "A NAND flash memory chip is a massive array of highly-regular memory cells that have been thoroughly characterized both in terms of aggregate/average behavior and localized defect rates." First, your point about SSD firmware complexity. Perhaps you are correct but I'd not be convinced until seeing the architecture and source code (it's not only SSDs, where the secrecy/lack of source code is a problem, it's a serious issue with many products and likely won't be resolved until AI is usefully deployed into reverse engineering it.) Clearly, you have considerably more faith in NAND technology than I do. That's fine, your objectives are likely very different to mine. I'll just say this, current tends in NAND technology where more and more bits are stored in smaller and smaller 'cells' crammed in more and more stacked layers and that each data bit consists of fewer and fewer bottled-up charges that eventually leak away despite quantum potentials trying to keep them in place, is not my idea of a reliable, permanent storage system. I've been recovering data from dead drives of both types for quite some time now so my views are mitigated by how difficult it can be. I'm not defending HDDs either except to say that as a more mature tech and given its storage architecture it is easier to do so. I'll also grant that recovering data from some HDDs is intractable but such instances are comparatively rare. Incidentally, I remember one such failure very well as it was so obviously connected with what I did. It was a Connor Peripherals 20MB drive—yes, it was a long while ago—in a desktop PC. I dropped a small book onto the table where the PC was and the drive instantly gave read errors. Despite strenuous efforts, its data was irrecoverable.

hammyhavoc 3y ago

Yes, I have suffered data loss from both HDDs and SSDs, paid for the recovery services available, and HDDs have always been the ones to see any degree of success. SSDs? You're probably screwed.

quags 3y ago
I’ve done three recoveries to a remote service for clients - the first failed and two which were ssd had a full success.m but all were ssds. Hdds I never sent to recovery because generally they were not complete failures (ie some bad blocks and a copy around was possible ) or the board could be replaced to fix it with the same model and firmware.
MichaelZuo 3y ago
Meaningful data recovery from a failed SSD could need tens of millions of dollars worth of equipment, depending on how it failed.
- CyberDildonics 3y ago
  Can you give a source for why you think that?

hilbert42 3y ago

"I wonder if SSD firmware still needs to mature to the point..." This is a major issue, I've had almost brand new SSDs fail catastrophically without warning. In one instance I'd gone to great trouble to create an organized data disk for archiving and it failed just as I was about to back it up, and nothing I did could get it to even recognize the SATA interface let alone give me access to the data. This is most annoying, first, why does the interface have to also die when something goes wrong with the memory? Second, can't the firmware at least tell us what's wrong with the memory? (Given the interface is essentially the smallest part of the drive's real estate it's unlikely the bit that fails, similarly the firmware would be second smallest—even then, why isn't redundancy built into drives for these two vital components (two of each)?) So what's going on? To maintain the integrity of our data we not only need these issues solved but also we need to know much more about the architecture of SSD drives. As users, we ought to pay much more attention to the fact that much of this problem stems from the inordinate amount of secrecy SSD manufacturers adopt. We really do need some new manufacturer to come to market with a drive based on data integrity, its selling point would be 'it's open, here's what we do and here are our tools to help you recover your data'. There's also the longevity problem that needs resolution, I've put SSD as backups into storage only to come back several years later to find the data missing. As it is, the data on HDDs is ephemeral enough, but relying on a 'well' of electrons trapped in 'glass' is not only risky but almost foolhardy. If it had not been for the speed and convenience of SSDs we'd never have contemplated using them in the first instance. Before we scrap HDDs we really do need much more reliable data storage than SSDs, especially so for archival data. It remains to be seen what eventuates, but whatever it is it should be based on a technology (physics) that's inherently reliable and stable, crystals for instance.

wtallis 3y ago
> This is most annoying, first, why does the interface have to also die when something goes wrong with the memory? I think you've read too many horror stories about unreliability of flash memory and are now blaming the flash memory for everything that goes wrong with an SSD even in instances that look very likely to be a firmware bug rather than a direct result of flash memory failing. Flash memory is error-prone, but largely in predictable ways. SSD firmware is a pile of overcomplicated software developed in a hurry and squeezed into an embedded system with limited resources. Do you really think the former is more likely to randomly cause catastrophic failures? > Given the interface is essentially the smallest part of the drive's real estate it's unlikely the bit that fails, similarly the firmware would be second smallest—even then, why isn't redundancy built into drives for these two vital components (two of each)? Maybe you're judging "smallest" in terms of transistor count or something, but your metric should be complexity. A NAND flash memory chip is a massive array of highly-regular memory cells that have been thoroughly characterized both in terms of aggregate/average behavior and localized defect rates. By contrast, SSD firmware is among other things implementing at least half the functionality of a filesystem, and the host interface of the SSD has to interact with third-party components that only loosely comply with protocol specifications that are themselves sometimes shockingly vague or silent on important questions.
- hilbert42 3y ago
  "think you've read too many horror stories about unreliability of flash memory..." Given my collection of dozens of dead SSDs I have my own horror stories, my comments are based on painful experience. In most instances whenever I've had a HDD fail—and that's many over the years—I have been able to recover the data but I cannot say that about SSDs or USB thumb drives as they mostly fail catastrophically and without warning—and the data is irrecoverable. Moreover, with SSDs, in cases where data has been accidentally erased, wear leveling often mitigates against successful recovery. I've lost far more data to SSDs than I ever have to HDDs yet I've only been using them for about one third as long. I've now reached the point where I don't bother to recover data from a dead SSD as I know it'll likely be a futile effort. That's still not so with HDDs but there are exceptions. Moreover, whether it's hardware failure or accidental data erasure, ALL SSD and USB thumb drive manufacturers either provide no recovery tools or if they do then they are inadequate almost to the point of being useless (for instance, SanDisk's recovery tool is hopeless, I've just unsuccessfully tried it on its own 256GB SD cards). My contention is that if the firmware architectures of these devices were actually designed to be aware of file structures then recovery would be easier. Essentially, they're still not user-friendly and manufacturers have made little or no effort to rectify the problems. "SSD firmware is a pile of overcomplicated software developed in a hurry and squeezed into an embedded system with limited resources." Limited resources, why? A 500GB SSD is orders of magnitude larger than even the most bloated firmware would occupy. (The world' most bloated software Windows only takes about 7GB!) If SSD firmware is developed in such a hurry and thus jeopardizes users' data then I would contend the products are not fit for purpose. The fact remains that SSDs are intentionally made as 'black boxes' by their manufacturers and I'd maintain that this is highly unsatisfactory as far as users are concerned. We ought to be long past the point where users' data are put at risk by shoddy or limited design efforts and or the desire of manufacturers to keep their designs secret. Nowadays there's too much at stake just to hold the world's data inside black boxes that no one understands—ones that not even manufacturers can recover (or are willing to recover) data from when they fail. We users deserve much better. [In saying that I'm no apologist for HDD manufacturers who adopt similar tactics.] "A NAND flash memory chip is a massive array of highly-regular memory cells that have been thoroughly characterized both in terms of aggregate/average behavior and localized defect rates." First, your point about SSD firmware complexity. Perhaps you are correct but I'd not be convinced until seeing the architecture and source code (it's not only SSDs, where the secrecy/lack of source code is a problem, it's a serious issue with many products and likely won't be resolved until AI is usefully deployed into reverse engineering it.) Clearly, you have considerably more faith in NAND technology than I do. That's fine, your objectives are likely very different to mine. I'll just say this, current tends in NAND technology where more and more bits are stored in smaller and smaller 'cells' crammed in more and more stacked layers and that each data bit consists of fewer and fewer bottled-up charges that eventually leak away despite quantum potentials trying to keep them in place, is not my idea of a reliable, permanent storage system. I've been recovering data from dead drives of both types for quite some time now so my views are mitigated by how difficult it can be. I'm not defending HDDs either except to say that as a more mature tech and given its storage architecture it is easier to do so. I'll also grant that recovering data from some HDDs is intractable but such instances are comparatively rare. Incidentally, I remember one such failure very well as it was so obviously connected with what I did. It was a Connor Peripherals 20MB drive—yes, it was a long while ago—in a desktop PC. I dropped a small book onto the table where the PC was and the drive instantly gave read errors. Despite strenuous efforts, its data was irrecoverable.