4 ms·

I was under the impression that the shred command was pretty hard on SSDs. Did that change?

LinuxBender 4y ago
It would be as hard as writing the file {n} number of times again. So if one created a 1MB log file and then did a 3 pass wipe that would write 3MB. Number of passes would really just depend on the system being worked on. If PCI I would expect 7 passes and I am sure that would just be factored into the cost of the servers for that environment. Shred also isn't perfect as it has no concept of the file systems journal and does not clean that data but I think it is still good practice for sensitive data, in addition to filesystem or file level encryption on systems with highly sensitive data. With sensitive data one must look at the value of the data vs the value of the SSD. If I go out of my way to extend the life of my SSD, do I risk losing $5billion? That is how I factor in whether or not extra wear-and-tear on a SSD makes sense.
- electroly 4y ago
  Shred doesn't work at all on SSDs because wear leveling will spray the writes all over the drive, rather than overwriting the blocks you intended to. Your random bytes will end up in new blocks, not overwriting the original blocks. Trying to shred individual files is a totally pointless exercise on SSDs. It isn't doing what you want it to do.
  - LinuxBender 4y ago
    I've also seen people say that and to instead use "blkdiscard -sz" but that also does not write to specific blocks. I still can not find a definitive source saying that shred is not able to write to specific blocks on a SSD and it is not clear to me that wear leveling actually prevents shred from writing to specific blocks. But I agree that I have seen people say this repeatedly on StackExchange, ServerFault and Reddit. Either way one should also use filesystem and file level encryption for sensitive files and encrypt sensitive attributes inside databases. Swap must also be encrypted as it may contain sensitive data. Datacenter bad-drive mishaps do happen as in not shredding the physical drive as a few government and military agencies have recently been embarrassed by. I do agree with pinkorchid that ultimately drives should be destroyed physically.
    - electroly 4y ago
      Wear leveling prevents anyone from writing to specific blocks. It decouples the OS's block numbering from the physical blocks. That indirection allows it to distribute writes evenly across the drive even if the OS is writing to consecutive blocks (or the same block). This behavior is fundamental; it's what wear leveling is and why it exists. Wear leveling exists to prevent you from writing to the same block repeatedly, which is exactly what you're trying to do with shred. I'm not sure what other source you're looking for, but I hope you find it. Perhaps, at least, the number of people "on StackExchange, ServerFault and Reddit" and now also HackerNews telling you the same thing is sufficient evidence to stop spreading the idea that shred might work on SSDs. Do what you want on your own systems, of course, but this is dangerously misleading guidance to be offering on an open forum.
      - sgc 4y ago
        Does that mean that partitions on an SSD therefore do not affect the long term wear on the SSD?
      - LinuxBender 4y ago
        So my concern with following this logic is that such tools that are writing to specific block locations would then be wiping unrelated inodes depending on the low level logic. I tried emailing Colin but his last known email is invalid now. He may be working for the NSA so I probably won't be able to reach him. Some people there are responsive, some are not and a few are on here. So even if wear leveling prevents overwriting a file then such tools should in theory be a risk of data corruption. If this is the case then the tool should be updated to detect if the target is an SSD and abort with a scary message. Perhaps another route is to reach out to the coreutils team.
        10987654321 4y ago
        > writing to specific block locations would then be wiping unrelated inodes depending on the low level logic It won't wipe unrelated files as far as the file system sees it, but it might overwrite some previously discarded internal blocks. "inodes" is a too high-level concept in this context. You can read more about it on wikipedia which might be an authorative enough source for you? https://en.wikipedia.org/wiki/Wear_leveling https://en.wikipedia.org/wiki/Wear_leveling Also note that most ssds actually have more internal blocks available than what they present to the host device so that they can have a "cache" to be able to move things around internally, and also so they can mark certain positions as "bad" when writes to one internal block start failing and still operate properly.
  - deleted 4y ago
    [deleted]

It would be as hard as writing the file {n} number of times again. So if one created a 1MB log file and then did a 3 pass wipe that would write 3MB. Number of passes would really just depend on the system being worked on. If PCI I would expect 7 passes and I am sure that would just be factored into the cost of the servers for that environment. Shred also isn't perfect as it has no concept of the file systems journal and does not clean that data but I think it is still good practice for sensitive data, in addition to filesystem or file level encryption on systems with highly sensitive data. With sensitive data one must look at the value of the data vs the value of the SSD. If I go out of my way to extend the life of my SSD, do I risk losing $5billion? That is how I factor in whether or not extra wear-and-tear on a SSD makes sense.

electroly 4y ago
Shred doesn't work at all on SSDs because wear leveling will spray the writes all over the drive, rather than overwriting the blocks you intended to. Your random bytes will end up in new blocks, not overwriting the original blocks. Trying to shred individual files is a totally pointless exercise on SSDs. It isn't doing what you want it to do.
- LinuxBender 4y ago
  I've also seen people say that and to instead use "blkdiscard -sz" but that also does not write to specific blocks. I still can not find a definitive source saying that shred is not able to write to specific blocks on a SSD and it is not clear to me that wear leveling actually prevents shred from writing to specific blocks. But I agree that I have seen people say this repeatedly on StackExchange, ServerFault and Reddit. Either way one should also use filesystem and file level encryption for sensitive files and encrypt sensitive attributes inside databases. Swap must also be encrypted as it may contain sensitive data. Datacenter bad-drive mishaps do happen as in not shredding the physical drive as a few government and military agencies have recently been embarrassed by. I do agree with pinkorchid that ultimately drives should be destroyed physically.
  - electroly 4y ago
    Wear leveling prevents anyone from writing to specific blocks. It decouples the OS's block numbering from the physical blocks. That indirection allows it to distribute writes evenly across the drive even if the OS is writing to consecutive blocks (or the same block). This behavior is fundamental; it's what wear leveling is and why it exists. Wear leveling exists to prevent you from writing to the same block repeatedly, which is exactly what you're trying to do with shred. I'm not sure what other source you're looking for, but I hope you find it. Perhaps, at least, the number of people "on StackExchange, ServerFault and Reddit" and now also HackerNews telling you the same thing is sufficient evidence to stop spreading the idea that shred might work on SSDs. Do what you want on your own systems, of course, but this is dangerously misleading guidance to be offering on an open forum.
    - sgc 4y ago
      Does that mean that partitions on an SSD therefore do not affect the long term wear on the SSD?
    - LinuxBender 4y ago
      So my concern with following this logic is that such tools that are writing to specific block locations would then be wiping unrelated inodes depending on the low level logic. I tried emailing Colin but his last known email is invalid now. He may be working for the NSA so I probably won't be able to reach him. Some people there are responsive, some are not and a few are on here. So even if wear leveling prevents overwriting a file then such tools should in theory be a risk of data corruption. If this is the case then the tool should be updated to detect if the target is an SSD and abort with a scary message. Perhaps another route is to reach out to the coreutils team.
      - 10987654321 4y ago
        > writing to specific block locations would then be wiping unrelated inodes depending on the low level logic It won't wipe unrelated files as far as the file system sees it, but it might overwrite some previously discarded internal blocks. "inodes" is a too high-level concept in this context. You can read more about it on wikipedia which might be an authorative enough source for you? https://en.wikipedia.org/wiki/Wear_leveling https://en.wikipedia.org/wiki/Wear_leveling Also note that most ssds actually have more internal blocks available than what they present to the host device so that they can have a "cache" to be able to move things around internally, and also so they can mark certain positions as "bad" when writes to one internal block start failing and still operate properly.
- deleted 4y ago
  [deleted]