3 ms·

Hey, can you recommend a super-simple way to automatically anonymize customer data?

by tomcam 3y ago

mceachen 3y ago
I dunno, I read someplace that there’s no sure-fire way to do that.
webreac 3y ago
rm is a good start, but you also need to ensure the disks are not sold to anyone.
- bayindirh 3y ago
  When you enable TRIM, it's zeroed automatically, so you're 99% there with an rm.
  - bestham 3y ago
    No, the point of enabling TRIM is to not induce further write amplification. The zeroing is deferred until the block that is trimmed is over written. It is released from the indefinite garbage collection that exists in a pure SSD implementation that does not know what blocks are important. Some firmwares answers with nothing when requesting a TRIMed block. Others answer with the content of the block as it is on NAND. Either way, the bits are most likely still on NAND.
    - bayindirh 3y ago
      I think it's dependent on the GC scheme of the SSD in question. From what I have seen, some drives defer it until they'll be written, some zero at the moment the sectors are TRIMmed and some wait for idle + timeout amount of time before working on these sectors. I have a couple of external SSDs and USB flash drives start to flash their busy lights after letting them idle for a couple of minutes. These "bursts of activity" generally is proportional to the I/O work they have done before the idle period. One of my flash drives did that GC/zeroing thing when it's ejected. After ejection and drive disconnect, its busy light flashed for some time depending on the work needs to be done. If you didn't hammer it when you used it, it did nothing, but if you did tons of work, it worked up to a minute or so after being ejected.
simlevesque 3y ago
The heat death of the universe
- tomcam 3y ago
  Why do I never think of these things
- Etherlord87 3y ago
  Source? I don't think physicists agree. For a while there was an idea getting popularity, that black holes erase information, but in the recent year I've seen some articles and youtube videos (didn't actually care to read/watch) with titles saying otherwise. To my understanding, the idea behind the heat death of the Universe is that nothing interesting happens anymore, not that there is absolutely nothing.
  - tomcam 3y ago
    > nothing interesting happens anymore, not that there is absolutely nothing. Sorry, are you talking about the heat death of the universe or the last five Marvel movies?
    - TeMPOraL 3y ago
      Yes sirs, they are the same picture.
  - aik 3y ago
    1. “The heat death of the universe” is my favorite HN comment of the decade. 2. The heat death of the universe does not mean one gigantic black hole. I’m just a hobbyist but my understanding of the theory is that black holes will continue to form, but through Hawking radiation, they eventually radiate out all their energy until it is all dispersed, ultimately leading to uniformity across the entire universe, max entropy, where “work” can no longer take place. (It is an interesting question then whether information is actually destroyed through Hawking radiation?)
    - 3y ago
aquir 3y ago
Black hole and wait for radiate some out as Hawking Radiation
- uKVZe85V 3y ago
  Not even certain this will be enough. Hawking radiation encodes the information that was thrown in. Sure at present it's a perfect mess, but who knows?
bastawhiz 3y ago
Microwave ovens, strong magnets (depending on the medium), shredders, blast furnaces, the hydraulic press from that YouTube channel, firing it into the sun...
Sohcahtoa82 3y ago
Yeah, right after I figure out how to make a regex match open tags except XHTML self-contained tags. If you're one of the lucky 10,000 that don't know the reference: https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags https://stackoverflow.com/questions/1732348/regex-match-open... And if you're one of the lucky 10,000 that don't know the other reference: https://xkcd.com/1053/ https://xkcd.com/1053/
pyuser583 3y ago
Step 1: generate uuid. Step 2: encrypt customer data. Step 3: associate encrypted data with uuid; keep uuid in encrypted if metadata. Step 4: use uuid instead of data. Obviously there are potential hiccups. But a lot of good can be done with encryption and metadata.

Hey, can you recommend a super-simple way to automatically anonymize customer data?

by tomcam 3y ago

Hey, can you recommend a super-simple way to automatically anonymize customer data?

mceachen 3y ago

I dunno, I read someplace that there’s no sure-fire way to do that.

webreac 3y ago

rm is a good start, but you also need to ensure the disks are not sold to anyone.

bayindirh 3y ago
When you enable TRIM, it's zeroed automatically, so you're 99% there with an rm.
- bestham 3y ago
  No, the point of enabling TRIM is to not induce further write amplification. The zeroing is deferred until the block that is trimmed is over written. It is released from the indefinite garbage collection that exists in a pure SSD implementation that does not know what blocks are important. Some firmwares answers with nothing when requesting a TRIMed block. Others answer with the content of the block as it is on NAND. Either way, the bits are most likely still on NAND.
  - bayindirh 3y ago
    I think it's dependent on the GC scheme of the SSD in question. From what I have seen, some drives defer it until they'll be written, some zero at the moment the sectors are TRIMmed and some wait for idle + timeout amount of time before working on these sectors. I have a couple of external SSDs and USB flash drives start to flash their busy lights after letting them idle for a couple of minutes. These "bursts of activity" generally is proportional to the I/O work they have done before the idle period. One of my flash drives did that GC/zeroing thing when it's ejected. After ejection and drive disconnect, its busy light flashed for some time depending on the work needs to be done. If you didn't hammer it when you used it, it did nothing, but if you did tons of work, it worked up to a minute or so after being ejected.

simlevesque 3y ago

The heat death of the universe

tomcam 3y ago
Why do I never think of these things
Etherlord87 3y ago
Source? I don't think physicists agree. For a while there was an idea getting popularity, that black holes erase information, but in the recent year I've seen some articles and youtube videos (didn't actually care to read/watch) with titles saying otherwise. To my understanding, the idea behind the heat death of the Universe is that nothing interesting happens anymore, not that there is absolutely nothing.
- tomcam 3y ago
  > nothing interesting happens anymore, not that there is absolutely nothing. Sorry, are you talking about the heat death of the universe or the last five Marvel movies?
  - TeMPOraL 3y ago
    Yes sirs, they are the same picture.
- aik 3y ago
  1. “The heat death of the universe” is my favorite HN comment of the decade. 2. The heat death of the universe does not mean one gigantic black hole. I’m just a hobbyist but my understanding of the theory is that black holes will continue to form, but through Hawking radiation, they eventually radiate out all their energy until it is all dispersed, ultimately leading to uniformity across the entire universe, max entropy, where “work” can no longer take place. (It is an interesting question then whether information is actually destroyed through Hawking radiation?)
  - 3y ago

aquir 3y ago

Black hole and wait for radiate some out as Hawking Radiation

uKVZe85V 3y ago
Not even certain this will be enough. Hawking radiation encodes the information that was thrown in. Sure at present it's a perfect mess, but who knows?

bastawhiz 3y ago

Microwave ovens, strong magnets (depending on the medium), shredders, blast furnaces, the hydraulic press from that YouTube channel, firing it into the sun...

Sohcahtoa82 3y ago

Yeah, right after I figure out how to make a regex match open tags except XHTML self-contained tags. If you're one of the lucky 10,000 that don't know the reference: https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags https://stackoverflow.com/questions/1732348/regex-match-open... And if you're one of the lucky 10,000 that don't know the other reference: https://xkcd.com/1053/ https://xkcd.com/1053/

pyuser583 3y ago

Step 1: generate uuid. Step 2: encrypt customer data. Step 3: associate encrypted data with uuid; keep uuid in encrypted if metadata. Step 4: use uuid instead of data. Obviously there are potential hiccups. But a lot of good can be done with encryption and metadata.