4 ms·

Repeat after me: There's no sure-fire way to automatically anonymize arbitrary customer data. There's no sure-fire way to automatically anonymize arbitrary cu

Repeat after me: There's no sure-fire way to automatically anonymize arbitrary customer data. There's no sure-fire way to automatically anonymize arbitrary customer data. There's no sure-fire way to automatically anonymize arbitrary customer data. You can't anonymize it by hand. You can't anonymize it by machine. If you're taking blobs of customer data and dumping it into a machine, customer data can come out of the machine. Neither humans nor machines are error-free. If you believe DocuSign is going to do a perfect job, I've got a bridge to sell you.

coding123 3y ago
Meh, so it puts Robert Picardo Stewart somewhere.
- bastawhiz 3y ago
  Or it puts your name and the name of one of your confidential clients in someone else's contract. Or it oopsies your name and salary into a job offer for someone else. Or it sees the name of your ex wife and hallucinates details about your assets based on your previous divorce papers and she sues you. I can come up with these all day.
  - wmidwestranger 3y ago
    AI as the system copying notes for the test from another system. Nothing in there is anywhere near impossible unless the system doesn't have the data.
  - xeyownt 3y ago
    And? If reliability is zero, so is the risk.
    - bastawhiz 3y ago
      Just because my ex wife might incorrectly sue me doesn't mean it's not an incredible pain in the ass. Just because someone incorrectly got a contract that hilariously includes all the private details of my employment doesn't mean I won't be royally pissed. The risk is "I don't fucking want this to happen". So no, it's not zero.
Der_Einzige 3y ago
What happened to differential privacy?
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- saulrh 3y ago
  It doesn't work on arbitrary data. Hand-crafted metrics on structured data, yeah. Free text, no. Data that you don't know anything about, also not really.
- nl 3y ago
  It works. But understand what you are doing: you are adding noise to individual's data to make them less identifiable within a specified set of data. The amount of noise you need to add depends on the size of the set they are within. The OP is also right: For arbitrary customer data (within arbitrary sets of data) you can't guarantee anonymization. There are also multi-party computation systems, homomorphic encryption and federated learning approaches that all provide different approaches for parts of the private machine learning problem. It's hard.
  - bigiain 3y ago
    Worth investigation "k-anonymity" as well. https://en.wikipedia.org/wiki/K-anonymity https://en.wikipedia.org/wiki/K-anonymity k-anonymity is an attempt to solve the problem "Given person-specific field-structured data, produce a release of the data with scientific guarantees that the individuals who are the subjects of the data cannot be re-identified while the data remain practically useful." One serious problem with that approach is that no matter how well you k-anonymise your data set, someone else's dataset may deanonymise yours.
tomcam 3y ago
Hey, can you recommend a super-simple way to automatically anonymize customer data?
- mceachen 3y ago
  I dunno, I read someplace that there’s no sure-fire way to do that.
- webreac 3y ago
  rm is a good start, but you also need to ensure the disks are not sold to anyone.
  - bayindirh 3y ago
    When you enable TRIM, it's zeroed automatically, so you're 99% there with an rm.
    - bestham 3y ago
      No, the point of enabling TRIM is to not induce further write amplification. The zeroing is deferred until the block that is trimmed is over written. It is released from the indefinite garbage collection that exists in a pure SSD implementation that does not know what blocks are important. Some firmwares answers with nothing when requesting a TRIMed block. Others answer with the content of the block as it is on NAND. Either way, the bits are most likely still on NAND.
      - bayindirh 3y ago
        I think it's dependent on the GC scheme of the SSD in question. From what I have seen, some drives defer it until they'll be written, some zero at the moment the sectors are TRIMmed and some wait for idle + timeout amount of time before working on these sectors. I have a couple of external SSDs and USB flash drives start to flash their busy lights after letting them idle for a couple of minutes. These "bursts of activity" generally is proportional to the I/O work they have done before the idle period. One of my flash drives did that GC/zeroing thing when it's ejected. After ejection and drive disconnect, its busy light flashed for some time depending on the work needs to be done. If you didn't hammer it when you used it, it did nothing, but if you did tons of work, it worked up to a minute or so after being ejected.
xyst 3y ago
We already know this. How about you tell the government and public officials.
Ferret7446 3y ago
There is, depending on the type, format, and volume of data. Simply classify data into buckets for each value of each variable such that each unique combination of buckets for every variable contains at least N items for a suitably large number of N. It is then mathematically impossible to match an individual data point to any set of original data points less than N.
- robin_reala 3y ago
  I’ve read this three times now and I still don’t understand. Can you explain it in a different way?
  - 65a 3y ago
    https://en.wikipedia.org/wiki/K-anonymity https://en.wikipedia.org/wiki/K-anonymity Basically keeping only chunks of text which have are not unique and contributed by many users.
    - bastawhiz 3y ago
      > Given person-specific field-structured data That's not what this data is and that's broadly not what LLMs are trained on.
      - 65a 3y ago
        By chunking and binning the data by user contribution count, you are structuring it as described. Pretraining is often accomplished on similar chunked text. k-anonymity is not perfect though, or state of the art, the wikipedia explains several known attacks.
- tdudhhu 3y ago
  [flagged]
- pas 3y ago
  it's still leaking information. let's say there are 5-6 companies that all work in the mystical cloud-painting sector, you lump them together and publish data about the sector. before there was nothing available now there's an average. and obviously partial information is very useful for exactly those with some other partial information. this is why OSINT is very effective. it's possible to piece together enough constraints on these datasets that in the end the likelihood of a successful deanonymization is unexpectedly high.
theschreon 3y ago
There's no sure-fire way to make a human who is processing a document manually never leak the information they read.
- Etherlord87 3y ago
  Sure, but if there's a law forbidding you to exceed a particular speed on the road, you can't do it anyway and say "you can't be perfectly safe anyway". The analogy here is: there are laws regarding confidentiality that probably were broken here.
  - theschreon 3y ago
    I agree. But at the same time, insisting on 100% certainty/safety would mean to not do anything at all and stick to the status quo forever. It boils down to cost-benefit-calculations. While I agree that it is unacceptable to use customer data without consent (as suggested by OP's post), I disagree with the implicit assumption behind the comment that I responded to: Namely the implicit assumption that human/biological intelligence/agents are somehow superior to artificial intelligence/agents in face of secrecy/confidentiality. It boils down to the question whether it's possible or not to create algorithms that outperform humans in tasks involving secrecy and confidentiality. While I can't think of any reason why that should not be possible in general, I agree that the current SOTA of generative LLMs is not sufficient for that. Is throwing lots and lots of data and RLHF training on an LLM enough in order to make the probability of customer data leaks small enough to be acceptable? I don't know. But I don't trust MBAs who salivate with dollar signs in their eyes to know either. And I fear that their lack of technical understanding will lead to bad decisions. And I fear those might lead to scandals that make Gemini's weird biases in image generation pale in comparison.
    - ptx 3y ago
      > It boils down to cost-benefit-calculations. Yes, the user bears the cost when their confidential data is leaked and the company derives the economic benefit of mishandling it, which is why this keeps happening.
SAI_Peregrinus 3y ago
To be pedantic, you can trivially anonymize arbitrary data by returning a constant. You can't automatically anonymize arbitrary customer data and have the result be useful.

coding123 3y ago

Meh, so it puts Robert Picardo Stewart somewhere.

bastawhiz 3y ago
Or it puts your name and the name of one of your confidential clients in someone else's contract. Or it oopsies your name and salary into a job offer for someone else. Or it sees the name of your ex wife and hallucinates details about your assets based on your previous divorce papers and she sues you. I can come up with these all day.
- wmidwestranger 3y ago
  AI as the system copying notes for the test from another system. Nothing in there is anywhere near impossible unless the system doesn't have the data.
- xeyownt 3y ago
  And? If reliability is zero, so is the risk.
  - bastawhiz 3y ago
    Just because my ex wife might incorrectly sue me doesn't mean it's not an incredible pain in the ass. Just because someone incorrectly got a contract that hilariously includes all the private details of my employment doesn't mean I won't be royally pissed. The risk is "I don't fucking want this to happen". So no, it's not zero.

Der_Einzige 3y ago

What happened to differential privacy?

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
saulrh 3y ago
It doesn't work on arbitrary data. Hand-crafted metrics on structured data, yeah. Free text, no. Data that you don't know anything about, also not really.
nl 3y ago
It works. But understand what you are doing: you are adding noise to individual's data to make them less identifiable within a specified set of data. The amount of noise you need to add depends on the size of the set they are within. The OP is also right: For arbitrary customer data (within arbitrary sets of data) you can't guarantee anonymization. There are also multi-party computation systems, homomorphic encryption and federated learning approaches that all provide different approaches for parts of the private machine learning problem. It's hard.
- bigiain 3y ago
  Worth investigation "k-anonymity" as well. https://en.wikipedia.org/wiki/K-anonymity https://en.wikipedia.org/wiki/K-anonymity k-anonymity is an attempt to solve the problem "Given person-specific field-structured data, produce a release of the data with scientific guarantees that the individuals who are the subjects of the data cannot be re-identified while the data remain practically useful." One serious problem with that approach is that no matter how well you k-anonymise your data set, someone else's dataset may deanonymise yours.

tomcam 3y ago

Hey, can you recommend a super-simple way to automatically anonymize customer data?

mceachen 3y ago
I dunno, I read someplace that there’s no sure-fire way to do that.
webreac 3y ago
rm is a good start, but you also need to ensure the disks are not sold to anyone.
- bayindirh 3y ago
  When you enable TRIM, it's zeroed automatically, so you're 99% there with an rm.
  - bestham 3y ago
    No, the point of enabling TRIM is to not induce further write amplification. The zeroing is deferred until the block that is trimmed is over written. It is released from the indefinite garbage collection that exists in a pure SSD implementation that does not know what blocks are important. Some firmwares answers with nothing when requesting a TRIMed block. Others answer with the content of the block as it is on NAND. Either way, the bits are most likely still on NAND.
    - bayindirh 3y ago
      I think it's dependent on the GC scheme of the SSD in question. From what I have seen, some drives defer it until they'll be written, some zero at the moment the sectors are TRIMmed and some wait for idle + timeout amount of time before working on these sectors. I have a couple of external SSDs and USB flash drives start to flash their busy lights after letting them idle for a couple of minutes. These "bursts of activity" generally is proportional to the I/O work they have done before the idle period. One of my flash drives did that GC/zeroing thing when it's ejected. After ejection and drive disconnect, its busy light flashed for some time depending on the work needs to be done. If you didn't hammer it when you used it, it did nothing, but if you did tons of work, it worked up to a minute or so after being ejected.

xyst 3y ago

We already know this. How about you tell the government and public officials.

Ferret7446 3y ago

There is, depending on the type, format, and volume of data. Simply classify data into buckets for each value of each variable such that each unique combination of buckets for every variable contains at least N items for a suitably large number of N. It is then mathematically impossible to match an individual data point to any set of original data points less than N.

robin_reala 3y ago
I’ve read this three times now and I still don’t understand. Can you explain it in a different way?
- 65a 3y ago
  https://en.wikipedia.org/wiki/K-anonymity https://en.wikipedia.org/wiki/K-anonymity Basically keeping only chunks of text which have are not unique and contributed by many users.
  - bastawhiz 3y ago
    > Given person-specific field-structured data That's not what this data is and that's broadly not what LLMs are trained on.
    - 65a 3y ago
      By chunking and binning the data by user contribution count, you are structuring it as described. Pretraining is often accomplished on similar chunked text. k-anonymity is not perfect though, or state of the art, the wikipedia explains several known attacks.
tdudhhu 3y ago
[flagged]
pas 3y ago
it's still leaking information. let's say there are 5-6 companies that all work in the mystical cloud-painting sector, you lump them together and publish data about the sector. before there was nothing available now there's an average. and obviously partial information is very useful for exactly those with some other partial information. this is why OSINT is very effective. it's possible to piece together enough constraints on these datasets that in the end the likelihood of a successful deanonymization is unexpectedly high.

theschreon 3y ago

There's no sure-fire way to make a human who is processing a document manually never leak the information they read.

Etherlord87 3y ago
Sure, but if there's a law forbidding you to exceed a particular speed on the road, you can't do it anyway and say "you can't be perfectly safe anyway". The analogy here is: there are laws regarding confidentiality that probably were broken here.
- theschreon 3y ago
  I agree. But at the same time, insisting on 100% certainty/safety would mean to not do anything at all and stick to the status quo forever. It boils down to cost-benefit-calculations. While I agree that it is unacceptable to use customer data without consent (as suggested by OP's post), I disagree with the implicit assumption behind the comment that I responded to: Namely the implicit assumption that human/biological intelligence/agents are somehow superior to artificial intelligence/agents in face of secrecy/confidentiality. It boils down to the question whether it's possible or not to create algorithms that outperform humans in tasks involving secrecy and confidentiality. While I can't think of any reason why that should not be possible in general, I agree that the current SOTA of generative LLMs is not sufficient for that. Is throwing lots and lots of data and RLHF training on an LLM enough in order to make the probability of customer data leaks small enough to be acceptable? I don't know. But I don't trust MBAs who salivate with dollar signs in their eyes to know either. And I fear that their lack of technical understanding will lead to bad decisions. And I fear those might lead to scandals that make Gemini's weird biases in image generation pale in comparison.
  - ptx 3y ago
    > It boils down to cost-benefit-calculations. Yes, the user bears the cost when their confidential data is leaked and the company derives the economic benefit of mishandling it, which is why this keeps happening.

SAI_Peregrinus 3y ago

To be pedantic, you can trivially anonymize arbitrary data by returning a constant. You can't automatically anonymize arbitrary customer data and have the result be useful.