11 ms·
The GDPR blog post
- meta_AU 8y agoIs there not any issue with having a hashed version of the email, given the entropy of an email address is quite small?
- tialaramex 8y agoThe practical entropy of email addresses is indeed pretty small, lots of them are going to be first.last@company.example and a bunch more end in gmail.com or another popular provider. If you can accept some level of false positives you could make the hash too narrow to be able to usefully reverse it. For example if only sixty people will ever subscribe or refuse to subscribe,a 24-bit hash is plenty to reject mistaken attempts to subscriber somebody who doesn't want in, but good luck guessing which GMail user is "2ca24b". Another problem is, what if the email address changes hands - maybe even the whole email domain changed ownership. You probably need a way for people to change their minds, as that then also covers the case where the person behind the address changed.
- krageon 8y agoIt cannot be reversed without a significant amount of effort (really, even when you say the "entropy is quite small" it's not actually as small as you would think) and is therefore probably reasonable. Worst case a regulating body will tell you that no, they do not think "this will take 1-10 years to reverse" is quite good enough and then you can work with them on a solution that would be good enough.
- bcoates 8y agoMinutes to days to reverse almost the whole list, depending on budget. It's not a real obstruction except to casual snooping.
- krageon 8y agoCould you walk me through how you come to that conclusion? I admit my estimate was very ballpark, but "minutes" seems so wildly out of line with what I think I must be making a mistake somewhere.
- bcoates 8y agoA single AWS GPU server can hash trial passwords on the order of 100 GH/s, which puts a pretty low ceiling on "hashcat as a service" rental costs. I'm assuming 10^12 tries per second is economical for any business. there are about a million words, including all likely spellings of all but the rarest first and last names, so all 1 or 2 word addresses, firstname.lastnames, etc. addresses are about 10^12. try those, plus short alphanumerics, for the 1000 most common email domains -> 10^15 addresses Throw in every name in public leak databases that doesn't meet those patterns as well. There's on the order of 1 million domains that are likely to be serving mail at all; try the billion most likely names for each of those for another 10^15. This should capture almost every email address that isn't an intentionally obfuscated one-off and adds up to less than an hour at 10^12/sec. There's a modest overhead to matching against a larger list but it shouldn't matter in practice
- jacquesm 8y agoA couple of hundred bucks spent on renting GPU instances can speed things up considerably.
- _nalply 8y agoPerhaps use bcrypt to be on the safe side. With correct bcrypt configuration brute-forcing gets infeasible.
- hvidgaard 8y agoMaybe, but they have a good reason to keep that data, and they even go out of their way to "hide it" the best they can using a one-way function. To save the information that a certain email address has explicitly withdrawn consent, they need to store it. The alternative is to send out a new email the next time someone adds then. I think the interpretation of GDPR this particular instance of information storing is still open, but they have done everything possible to keep it safe. Should the list of hashes be leaked, the best an adversary can realistically do is check known emails against the list of hashes.
- donkeyd 8y agoYup, this is exactly what GDPR is aimed at. They thought about what they need, why they need it and have it documented.
- lvh 8y agoYou're right, but there are safer constructions to do this. Maybe this kind of knowledge will get more popular now that GDPR is mandating it :) Active concern for me: GDPR will promote a bunch more homegrown looks-fine-but-actually-busted crypto schemes. I don't think GDPR will be used to enforce that even in the case of breach, and I'm not sure it should -- I think we should make better schemes available instead.
- talkingtab 8y agoWhat is the 'safer construction' to do this? I'm looking for ideas and trying to solve a problem. My understanding of the GDPR, which is very basic, supports the view that hashing email addresses is at least questionable. On the other hand, if an email list is a core function, de-spamming seems valid.
- zwily 8y agoAn appropriately tuned bloom filter would probably suffice.
- 8y ago
- lvh 8y agoYes. They don't quite define how the hash works in the post, but assuming it's something like SHA256(email), that's easy to enumerate. There are ways to do this better. Let's say that it's 1 party and you're trying to figure out if you've seen en email address before. (That's the case in the article, there are also schemes where you and another entity can figure out if you both saw any email addresses -- but that's not what we're discussing here.) We already know how to take relatively low entropy things and store them securely to see if you've seen them before, for password storage! However, password storage works a little differently. You _know_ which entry you're checking against because you have a secret (password) but also an identifier (user name) -- so you can recompute against the same random key. This randomization means attackers need to try every password for every user. This doesn't work for us, because we just have an email, but it's close. Three parts worth considering: KDF, PRF and truncation. Firstly, your (deterministic, for reasons mentioned above) PRF turns your low-entropy input into a higher-entropy key. But (again, for reasons mentioned above) attackers still just have to try every email should they compromise your database. You can fix that problem by also adding a PRF (pseudorandom function) that you rate-limit vigorously. Think of a PRF as a keyed hash -- the usual example is HMAC-SHA256. If you're capable of keeping PRF key material safe but might leak a database dump (not unreasonable), the PRF forces the attack to be online: an attacker can only validate guesses as long as they have access to the PRF, and the PRF comes with audit trails and rate limits. Finally, you can choose to truncate the output. Because the output space of your PRF will be much, much larger than the input space of email addresses, a match out of the PRF gives you almost perfect certainty that you've seen the email address before. That goes for you, and an attacker. If, let's say, you have another way to validate if you've seen the user before (but it's expensive, say, you have an encrypted offline dataset but it's AES-GCM'd and you can't afford to decrypt the entire thing every time), truncation gives you a neat way to _probabilistically_ say if you've seen an address before.
- hvidgaard 8y ago> You can fix that problem by also adding a PRF (pseudorandom function) that you rate-limit vigorously. Think of a PRF as a keyed hash -- the usual example is HMAC-SHA256. If you're capable of keeping PRF key material safe but might leak a database dump (not unreasonable), the PRF forces the attack to be online: an attacker can only validate guesses as long as they have access to the PRF, and the PRF comes with audit trails and rate limits. That particular part is assuming security through not knowing the implementation of the security models components, aka. security through obscurity. Rule no. 1 in security, always assume that the adversary knows exactly how everything is implemented and can do that for himself.
- Erwin 8y agoIt seems to me this counts only as pseudonymisation and not anonymization. While the hash is not directly readable, it's still reversible with additional information (such as a large list of email addresses and knowledge of the hashing algorithm). One of the GDPR notes says: > [p]ersonal data which have undergone pseudonymization, which could be attributed to a natural person by the use of additional information, should be considered to be information on an identifiable natural person” Consider that you are running some kind of controversial/embarassing site of sexual/political/other sensitive nature. You keep a hashes of people who once were users but unsubscribed or something like that. If that database is leaked, a user could re-hash list of political figures, celebrities or just some big list of well known email addresses and with this information find out they were users of this sensitive site. So to me it seems that pseudoanymized/hashed emails still count as PII and have to be treated as such.
- tallanvor 8y agoI honestly can't see this being something that would ever result in enforcement action. They have a legitimate business interest in not spamming someone if people try to sign you up multiple times, and since the email address is hashed, all they can use it for is to determine if they've sent you an invite before (and potentially when they did so, or when you declined the invitation). Maybe they could get in trouble if they also retain information on who is trying to send you invites and creating a graph and a shadow profile based on this type of information, but it sounds pretty clear that this isn't something they're doing or are interested in doing.
- TeMPOraL 8y agoA popup (probably what used to be cookie warning) on Medium says: > Medium uses browser cookies to give you the best possible experience. To make Medium work, we log user data and share it with processors. To use Medium, you must agree to our Privacy Policy. I must agree to logging user data and sharing it with processors? EDIT: come to think of it, it might be a new, GDPR-specific, dark pattern. I can use the site without clicking "I agree", and the existence of that button sort of implies the consent is not assumed. The wording of the message ("you must agree") is just trying to bait consent. EDIT2: I just read[0] that biggest sites in my country are treating closing the GDPR popup as giving consent to everything. This definitely does not sound as explicit, informed consent. I sincerely hope it'll land them in a world of hurt. -- [0] - (PL link) https://zaufanatrzeciastrona.pl/post/klikasz-x-w-komunikacie-o-rodo-wyrazasz-zgode-na-przetwarzanie-danych/ https://zaufanatrzeciastrona.pl/post/klikasz-x-w-komunikacie...
- jannes 8y agoSome complaints have been filed against Google and Facebook for this practice today: http://www.bbc.com/news/technology-44252327 http://www.bbc.com/news/technology-44252327
- isostatic 8y agoNope, the site works fine if you disable cookies. Once Facebook and Google fail I'm sure they'll be next.
- TeMPOraL 8y agoI didn't click "I Agree" anyway. If they processed the data, I guess they're in violation now. That said, it's not the first time I've seen something like that this week. I wonder if some companies aren't simply testing if they can get away with it.
- isostatic 8y agoYes, they say To make Medium work, we log user data and share it with processors. To use Medium, you must agree to our Privacy Policy, including cookie policy. However it seems to work just fine without cookies - when I load the site in lynx, and reject all cookies, it loads just fine.
- baq 8y agoI got a few dozen gdpr emails today, some from companies I didn't know existed. This law is a fantastic development for end users/consumers.
- TeMPOraL 8y agoYup. Today I opened my fridge wondering if I'll see a note about an updated privacy policy inside. It's ironic seeing that the law was in power for the last 2 years, but companies woke up only last week. A lot of those mails are only information, with no (clearly marked) link to a consent panel, so I assume that me ignoring them means they won't be allowed to spam me anymore.
- Cthulhu_ 8y agoThat is implied in some of the e-mails - that is, they're asking you for explicit opt-in permission to keep mailing you. Mind you the old required 'unsubscribe' link was often adequate, but I don't mind this either. The flood of emails is a nice reminder of how many services you're signed up with, too. Some even with multiple e-mail addresses.
- TeMPOraL 8y agoI was particularly surprised seeing names I don't even recognize. Turns out that some of my one-off on-line purchases were handled by companies with names completely different than the names of the shops they put on-line.
- varjag 8y agoYou're joking, but my fridge did hand me a GDPR notice in the morning: https://twitter.com/varjag/status/998496423019778048 https://twitter.com/varjag/status/998496423019778048
- TeMPOraL 8y agoWow. That's the funniest thing I've seen this week. Beats the schadenfreude I'm having with some of the IoT lightbulbs no longer working for EU customers - a problem which actually impacted a friend of mine. See: https://twitter.com/internetofshit/status/999619364541394944 https://twitter.com/internetofshit/status/999619364541394944. EDIT: 2 friends now. I wonder how many more bought those lightbulbs...
- maaaats 8y agoA colleague sent me this, lots of funny variants https://gdprhallofshame.com/ https://gdprhallofshame.com/
- zerostar07 8y agoWarning though: that site is not gdpr-compliant
- MitjaBezensek 8y agoWhy? Is there a company behind this webpage?
- Grue3 8y ago> sponsored spontaneously by the amazing Raygun.
- zerostar07 8y agodoesnt have to be a company. it processes data and it's not a strictly personal site, it's all over the internet in fact.
- jstanley 8y agoWhy is it not gdpr-compliant?
- zerostar07 8y agoprivacy policy, cookies, analytics, opt-in
- nicky0 8y agoI agree it's a rather nice site. So refreshing not to have to wade through privacy guff popups.
- merinowool 8y ago
- deleted 8y ago[deleted]
- y0ghur7_xxx 8y ago"To make Medium work, we log user data and share it with processors. To use Medium, you must agree to our Privacy Policy, including cookie policy." No Medium, I must NOT agree to your privacy policy and your cookie policy, because to use and share my data you need my FREE consent. AND you can NOT deny me reading an article without giving consent, because then the consent is not FREE, and it is NOT strictly necessary for the service. Medium: either you allow me to read blog posts on your webserver without FORCING me to allow you to collect my data, or you don't. Choose. But stop fucking annoying me with lying banners.
- dpwm 8y agoI think it's more likely that "to make Medium make money," they engage in tracking for advertising purposes. Medium works perfectly well for my purposes without that banner being displayed. I can open up developer tools and delete that node. If I don't click agree, does that mean that this information isn't collected? Because tracking cookies are still placed. Now what is interesting is that I don't remember being asked for consent for them to place a cookie to log the number of articles I read in a month as part of their sign-up funnel.
- TeMPOraL 8y ago> Now what is interesting is that I don't remember being asked for consent for them to place a cookie to log the number of articles I read in a month as part of their sign-up funnel. They could probably make this compliant by storing the counter in your local storage and never sending it anywhere - just having a piece of JS that essentially does: if(Storage.getItem("visits") > 6) { displaySignnupPopup(); }
- chopin 8y agoI block JS wherever I can, though.
- netsharc 8y agoAh, when I used to bother with Proxomitron (https://www.proxomitron.info/ https://www.proxomitron.info/), I could rewrite anything that went "over the wire" because it acts as a HTTP-proxy listening at localhost. I remember modifying Javascript lines so adding my own code was possible... One could add an SSL library and basically MITM HTTPS connections, but I never tried that.
- Grue3 8y ago> this is going to be another ridiculous Cookie Law Given the number of ugly popups I had to click within the last few days, it already is.
- SpecialistEMT 8y agoI never added this cookie law notice to any of our websites and apps and never had a single problem. We operate in the EU. Pretty small scale. We did nothing for gdpr.
- severine 8y agoEven in Europe (at least in Spain), mainstream journalists and pundits are generally misstating the effects and contents of GDPR. I wish not-so-hot takes like this are more widely read, and along with sane enforcing, contribute to the sorely needed education on these topics of the general population.
- kbsletten 8y agoSo, I'm really not trying to start a fight, please read this with curious intent. I personally don't really feel like keeping my email is a violation of my privacy. If they're not "processing" it (that feels like code for "data mining") is this really required? I mean my email address is literally a public means of contacting me. It's kind of fun that they decided to use a one-way hash, but this story doesn't make me feel like the internet has really been improved.
- TeMPOraL 8y agoThe problem is identification of physical persons. Your e-mail is public, but it also identifies you as a person. This is important, because it allows for correlating different data sets. Touch Surgery sounds like a honest company, so for them this was just some extra burden. But the same law prevents ShadyAdtechCo from getting datasets from several companies and joining them on e-mail column to build a profile of you, without your explicit, informed consent in several places.
- ensmotko 8y agoWouldn't then a hash of your email also identify you as a person? The companies can still build a profile of you if they just agree to use the same hashing function :/
- comex 8y agoOr even if ShadyAdtechCo just knows what the hashing function is, and has a list of plaintext email addresses to test against – perhaps obtained from one of the datasets they're joining against, or even from crawling the web.
- TeMPOraL 8y agoHashing should be done with salt for precisely that reason.
- 8y ago
- merinowool 8y agoSo far I have received over 300 GDPR emails. When I am supposed to read all this? How do I track it? How can I track what each company stores about me? Do I feel this in any way improved safety of my data? I don't think so.
- baq 8y agoIn theory, if you don't reply, all these companies should stop using your data and quite likely delete it. Sounds like improving safety for me.
- dominotw 8y agoNo this is not correct.
- frereubu 8y agoI think I may agree with you, but to say something like that you also need to say why you don't think it's correct, otherwise it's pretty unhelpful.
- dominotw 8y agosorry should've mentionted. This is discussed in other comments in this thread. https://news.ycombinator.com/item?id=17152939 https://news.ycombinator.com/item?id=17152939
- drusepth 8y agoIt sounds pretty tedious to sift through 300+ emails to find everyone you want to keep using your data and go through whatever process they have for replying.
- RugnirViking 8y agoWhy do you want them to keep using your data?
- pojkofd00m 8y agoLove how I got a popup asking me to sign up with a fb/ggle account, stating "To make Medium work, we log user data and share it with service providers."
- tomelders 8y agoI had a recruiter call me up with what I suspect was a made up role. At the end of the call he casually dropped in the line "ok, well, is it ok if I get back in touch when something more suitable comes in?" It was conspicuous. I asked is he'd asked me that because of GDPR. He said yes. I said no.
- appdrag 8y agoI have reported as SPAM all the GDPR emails i got from unknown companies, i never asked to receive all this shit (250 GDPR messages just this week)
- deleted 8y ago[deleted]
- matte_black 8y agoWe’re still not GDPR compliant and don’t plan to be. So far so good.
- SpecialistEMT 8y agoSame here.
- wdr1 8y ago> I would be very wary of a company who claims this legislation is onerous. ... and elsewhere ... > On the other hand it also was not very hard for us. We are not a creepy company. > This is not to say that preparing for GDPR didn’t take us 100s of hours. It did. A company who it didn't affect much, spent 100s of our hours? I think it would reasonable to call that onerous. The different & fair question would be if time was justified.
- bhelkey 8y agoAlso see: > We engaged a dedicated GDPR consultant
- tatersolid 8y agoThat’s the issue with GDPR, it’s that the regulatory burden for Facebook is the same as it is for a small company. At $dayjob we are at hundreds of thousands of dollars in staff time and legal fees (mostly updating and reviewing existing contracts). We don’t do anything shady with user data, and already have a robust data security program due to our industry. A family member’s small business which packages meats for the grocery is similarly burdened to the tune of hundreds of thousands. That’s a huge waste repeated millions of times over around the world. They could have just targeted this at the big web companies and Adtech firms with some simple qualifiers. This law isn’t really much good for consumers, but it’s very good for lawyers.
- y0ghur7_xxx 8y ago> A company who it didn't affect much, spent 100s of our hours? I think it would reasonable to call that onerous. 100 hours is 12.5 days. That is not much to protect your users data.
- mychael 8y agoAmerican entrepreneurs who are proponents of GDPR are experiencing some serious Stockholm Syndrome. Or possibly they're just faking their love for GDPR to virtue signal.