11 ms·
Amnesia – High-Accuracy Data Anonymization
- haffi112 6y agoDo these methods provide any guarantees or is it assumed that a human reviews the output to verify that it was properly anonymized?
- frankmcsherry 6y ago> Data anonymized with Amnesia are statistically guaranteed that they cannot be linked to the original data. It looks like (from other text on their site) they use variants on k-anonymity. This can prevent re-linking attacks back to the original data, but we've also known for a decade that this isn't especially strong. For example, two independent k-anonymous releases can unique identify everyone in the dataset[0]. [0]: https://dl.acm.org/doi/10.1145/1401890.1401926 https://dl.acm.org/doi/10.1145/1401890.1401926
- La1n 6y agoHowever that statistical guarantee also requires your pseudoidentifiers to be picked correctly, i.e. it only holds true if you select all variables the attacker could possibly know about a subject. I think that is the hard part here, it's not something I would recommend someone doing without a lot of research and experience for highly dimensional data.
- antisyzygy 6y agoRight. Even if you assume the worst-case-scenario there isn't some standard risk metric nor threshold to meet. I feel like differential privacy is the strongest definition we have, but it is also lacking from a practical standpoint. What does it mean to have N nats/bits of information gain from seeing the result of a query? How does this translate to my risk of a PII leak?
- specialist 6y agoThe very best case for Amnesia: veneer of GDPR compliance, maybe survive an audit. Clients are buying plausible deniability, hedging their liability. "Yes, we regrettably leaked sensitive data. But we followed all the rules. What more could we possibly do?"
- teraku 6y ago"Not save my data in illegal ways" Edit: "Not save my data"
- specialist 6y agoThis product is for orgs sharing data.
- Wronnay 6y agohttps://web.archive.org/web/20201202123130/https://amnesia.openaire.eu/ https://web.archive.org/web/20201202123130/https://amnesia.o...
- jacquesm 6y agoFair warning: anonymization is a hard problem. It is never easy, and you'd be surprised how many bits can leak out of what you thought was properly anonymized data. If you are using data for test purposes please use generated data, not anonymized data. This has the additional advantage that there is no potential path for live data to end up on a developers machine. added in edit: And also realize that just using a service such as this or similar actually increases the chances that you are leaking sensitive data, in fact uploading somewhere it is the very best way in which you could ensure that at some point there is a breach. Don't take the 'made easy' line for granted if possible ask the company for their audit reports and what measures they have in place to ensure that your data doesn't end up elsewhere, a company providing such a service or a chunk of software that does this is - of course - a massive target. The only way you would stay away from that extra risk is by running this on your premises on a machine that is not connected directly or indirectly to the outside world.
- ellimilial 6y agoThis. Alternatively look for an open-licensed dataset if one exists in your domain (e.g. using https://fairsharing.org https://fairsharing.org or, shameless plug, https://biokeanos.com https://biokeanos.com). With generated data you always add some assumptions, with more 'wild' data you have a chance to discover edge cases earlier.
- levesque 6y agoWhat about building machine learning models that make predictions on said data? Can't just test on fake data.
- jacquesm 6y agoThere is a German company that specializes in this: https://www.statice.ai/ https://www.statice.ai/ , and that too is a path that you should only walk if you fully understand the subject matter. So you could instead test on fake data that has the same (or as much as possible the same) statistical properties as the data that you would like to use.
- 6y ago
- m0nster 6y agoIf you're interested in tools such as Amnesia, you might also want to take a look at ARX, which supports much more anonymization methods, including Differential Privacy: https://arx.deidentifier.org https://arx.deidentifier.org https://github.com/arx-deidentifier/arx https://github.com/arx-deidentifier/arx Disclosure: I'm the main author of ARX.
- jacquesm 6y agoAnd has a huge advantage: it runs local.
- simongray 6y agoAmnesia doesn't...? Then what's the download link and github page for?
- braincoke 6y agoThe Readme is not user friendly and doesn't explain how to use or install the software. In the Github issue the developer points to an installation documentation that throws a 404 error. In my opinion, this source code was put out to appear "open" to the H2020 Programme, but they have no intention in actually helping users run the code locally.
- freshhawk 6y agoYeah, wow. This is one of the more aggressive examples of "open"-but-unrunnable I've ever seen. I think your opinion is dead on. And if this isn't intentional, then the level of effort is so low how could you trust the software with such a complex and subtle problem? It is clearly intentional, but they didn't even give themselves a plausibly deniable story.
- m0nster 6y agoTrue, Amnesia can also be run locally!
- jhfdbkofdcho 6y agoThis is not easy. This is not an easy thing to do and protect the data. Please don’t call this easy.
- kebman 6y agoSchools think it's enough anonymization to call pupils by their initials while mailing back and forth grades and sensitive data and characterizations about them. It's a rampant problem.
- privacylawthrow 6y agoFor data to be anonymous under GDPR, it is not enough that individuals cannot be identified from the anonymized data set. If individuals can be identified when the anonymous data set is compared with the source data set, the anonymized data is not "anonymous". For data to be truly anonymous under GDPR. there must be no other additional data that would allow for reidentification. If there is any other data that, when combined with the anonymous data, allows for reidentification, the data set is only pseudonymous and must be treated as personal data under GDPR.
- 3np 6y agoThat's the most concise and clear formulation of that I've seen so far. Thanks.
- smarx007 6y agohttps://en.wikipedia.org/wiki/K-anonymity#Methods_for_k-anonymization https://en.wikipedia.org/wiki/K-anonymity#Methods_for_k-anon...
- La1n 6y agok-anonymity is often only applied to "pseudoidentifiers", if you have the original dataset it'd be trivial to reverse k-anonymity applied that way. For example someone's blood pressure isn't considered an identifying variable, and would not need to be anonymised (should not too, to keep data utility high), however this would make linking against the original dataset trivial.
- smarx007 6y agoYou are right, time series data like BPM over time does not lend itself to anonymization nicely, the provider most likely will have to ask the user organizations what kind of measures (features) they need and return an average (if that's what the receiving organisation was after) that itself can be k-anonymized.
- 6y ago
- simongray 6y agohttps://amnesia.openaire.eu/download.html https://amnesia.openaire.eu/download.html > Amnesia Desktop Version 1.2.2 > Available for Windows and Linux > [ mockup that is supposed to resemble a macOS app ] :-P
- stingraycharles 6y agoObviously their designer is using macOS. :)
- stef25 6y agoWhen this service gets hacked and leaks data who pays the GDPR fine?
- lwigo 6y agoUse synthetic data instead? https://tonic.ai/ https://tonic.ai/
- m0nster 6y agoAlso with synthetic data, there is an inherent trade-off between privacy risks and the usefulness of the data produced. However, this trade-off can be of a different nature, resulting in advantages for synthetization, for example when protecting high-dimensional data.
- La1n 6y agoAre there good ways to measure the amount (original) subject level data that can be extracted from a synthetic dataset, or calculated risk of reidentification (which is nice and easy for k-anonymity (if your assumptions are valid))?
- antisyzygy 6y agoRisk of re-identification is hard to estimate. It's mostly because you have to assume some state of background knowledge. I.e. what fields does the adversary even know something about. If Im looking for a white male in new york city it's going to be harder to find my target than it would be if I also know their birth date and zip code.
- fbn79 6y agoThe website render with no scrollbar in my browser (Goolge Chrome 97 Linux). Very annoying
- adolph 6y agoPresentation: Ten Things I Have Learned about De-identification https://www.youtube.com/watch?v=56whNWFIicY&t=3055s https://www.youtube.com/watch?v=56whNWFIicY&t=3055s
- _alex_ 6y agoUpload my data to be anonymized? Nooooooooope
- ushkarev 6y agoHow much can you trust a third party to take your complete data set, full of sensitive information, and treat it correctly? How long will they store it for? Who can access it while it’s being processed? What gets output in their internal logs? They’re asking for a lot of blind faith
- T-A 6y agoFirst sentence in [1]: Amnesia is an application written in java and JavaScript and should be used locally for anonymizing a dataset. [1] https://amnesia.openaire.eu/about-documentation.html https://amnesia.openaire.eu/about-documentation.html
- okokok___ 6y agoWhy isn't this open sourced?
- dewey 6y agoIt is. https://github.com/dTsitsigkos/Amnesia https://github.com/dTsitsigkos/Amnesia
- okokok___ 6y agoyep just found that! thanks!
- inquirerofsorts 6y agoIs that after clicking the download button on the linked page in the OP?
- Geminidog 6y agoInsensitive name to people actually suffering from amnesia, (me).
- rocgf 6y agoI am sorry, but I disagree.
- Geminidog 6y agoYou disagree only because you don't suffer from this condition. You can't truly know how offensive it is without being the person who suffers from the condition. Does a white person truly know about how offensive it is to call an African American the n-word? No. He/She won't without becoming African American himself. It's the same story here.
- bigbubba 6y agoGive it a few minutes.
- Geminidog 6y agoGive what a few minutes?
- Toutouxc 6y agoI'm a grown up man with a shoe size of 40 (8). Is the 2018 movie Smallfoot insensitive towards me? I mean, I don't enjoy hearing about your suffering and I certainly don't wish it upon you, but why would you care about the name? Do you think that people somehow treat amnesia sufferers with less respect when they're exposed to the name in a context that's not explicitly serious and negative?
- Geminidog 6y agoThe same logic can be applied to black people and the n-word. Shall I name a product after the n-word with no harm intended or being explicitly serious or negative? Does that make it right? No. You may not be offended by the word "smallfoot" but that's just you. You can't make that judgment for black people and the n-word anymore then you can make that judgment for the word "amnesia" and the entire community of amnesiacs.
- tpoacher 6y agoI read this post by Adam Pearce and Ellen Jiang the other day and it was a great read: https://pair.withgoogle.com/explorables/anonymization/ https://pair.withgoogle.com/explorables/anonymization/
- jkh1 6y agoThere are tools to generate realistic patient data, for example Synthea: https://synthetichealth.github.io/synthea/ https://synthetichealth.github.io/synthea/
- makach 6y agoI will go for the superior product "dementia" that will be all mic drop