4 ms·

A dataset like this is going to have a bunch of personal information in it. When it’s distributed like this, how does that jive with regulations like GDPR? If a

by tobr 8y ago

A dataset like this is going to have a bunch of personal information in it. When it’s distributed like this, how does that jive with regulations like GDPR? If a HN user would like to delete all their comments, how would that request be forwarded to every user of this dataset?

damajor 8y ago
I support this question. Any comments ? NB: That's easy to downvote without commenting...
- espeed 8y ago
  The Internet is written in ink. You should assume that any and all public posts you make have already been replicated and archived by countless parties in countless ways by the time you hit delete. HN public postings are no different. The HN API [1] has been around in various forms for years and includes the same public data that's used to generate the public pages on the HN site, but rather than returning HTML pages designed for human consumption, the API returns the data in a JSON serialized form [2] designed for machine consumption [3]. When the HN API went live, it reduced the overhead and redundant work from all the programmers having to independently crawl and parse site. The HN BigQuery dataset is the same data returned by the HN API, Google just took the next step and did the work of loading it into BigQuery. [1] https://github.com/HackerNews/API https://github.com/HackerNews/API [2] https://en.wikipedia.org/wiki/Category:Data_serialization_formats https://en.wikipedia.org/wiki/Category:Data_serialization_fo... [3] https://en.wikipedia.org/wiki/Machine_to_machine https://en.wikipedia.org/wiki/Machine_to_machine
kolinko 8y ago
The data is already publicly available for use and reuse, just in a different form. Why would it be any different than the rules regarding the public/api display of the information?
- tobr 8y ago
  Maybe I’m misunderstanding what this is, is it not possible to query, download and process at a completely different scale than the API? If not, I suppose you might ask the same thing about the API.
petercooper 8y ago
If people are sharing their own PII in HN comments, they agreed to HN's T&Cs when signing up. Such T&Cs state (heavily trimmed for length): By uploading any User Content you hereby grant [..] a nonexclusive, worldwide [..] irrevocable license to [..] distribute [..] your User Content for any Y Combinator-related purpose in any form [..] Agreeing to the T&Cs and deliberately sharing information publicly covers the GDPR's "consent" lawful base. Even under GDPR this is not a situation where someone signed up for something else and then happen to have their personal data shared as a byproduct. They signed up to a site, agreed to T&Cs, and then explicitly and deliberately shared their personal data.
- CobrastanJorji 8y ago
  The GDPR's super complicated, but I'm pretty sure its Right of Erasure, and specifically Article 7(3), which gives data subjects the right to withdraw consent at any time and the clause “it shall be as easy to withdraw consent as to give it" trumps any ridiculous "irrevocable" license to distribute your content in any form forever. Also importantly, the GDPR requires that a controller not make a service conditional upon consent. Hacker News is likely not in compliance unless they make such data processing optional and require anyone interested to explicitly opt in. But, then again, I'm not a lawyer, and even if I were, actual lawyers don't seen to know what the hell the GDPR actually requires either.
  - dasil003 8y ago
    IAANAL, and I don't mean to single you out here, but this seemingly rational argument strikes me as subtle FUD. It's the type of argument that someone with a vested interest in collecting user data for profit might put forth in the hopes of polarizing the tech community and painting GDPR as out of touch with technical common sense. Again, I'm not accusing you of anything here, I'm just pointing out who benefits from framing the conversation this way. So far there is a lot of precedent for small operators shutting down their sites out of fear of GDPR, but there is actually no precedent for regulators having actually gone after small operators for anything resembling reasonable practices. The day may come where EU regulators try to crack down on forums for who are unwilling or unable to redact users messages post-facto, but we're nowhere close to that today and I don't see strong reason to believe that's where we're headed either.

by tobr 8y ago

damajor 8y ago

I support this question. Any comments ? NB: That's easy to downvote without commenting...

espeed 8y ago
The Internet is written in ink. You should assume that any and all public posts you make have already been replicated and archived by countless parties in countless ways by the time you hit delete. HN public postings are no different. The HN API [1] has been around in various forms for years and includes the same public data that's used to generate the public pages on the HN site, but rather than returning HTML pages designed for human consumption, the API returns the data in a JSON serialized form [2] designed for machine consumption [3]. When the HN API went live, it reduced the overhead and redundant work from all the programmers having to independently crawl and parse site. The HN BigQuery dataset is the same data returned by the HN API, Google just took the next step and did the work of loading it into BigQuery. [1] https://github.com/HackerNews/API https://github.com/HackerNews/API [2] https://en.wikipedia.org/wiki/Category:Data_serialization_formats https://en.wikipedia.org/wiki/Category:Data_serialization_fo... [3] https://en.wikipedia.org/wiki/Machine_to_machine https://en.wikipedia.org/wiki/Machine_to_machine

kolinko 8y ago

The data is already publicly available for use and reuse, just in a different form. Why would it be any different than the rules regarding the public/api display of the information?

tobr 8y ago
Maybe I’m misunderstanding what this is, is it not possible to query, download and process at a completely different scale than the API? If not, I suppose you might ask the same thing about the API.

petercooper 8y ago

If people are sharing their own PII in HN comments, they agreed to HN's T&Cs when signing up. Such T&Cs state (heavily trimmed for length): By uploading any User Content you hereby grant [..] a nonexclusive, worldwide [..] irrevocable license to [..] distribute [..] your User Content for any Y Combinator-related purpose in any form [..] Agreeing to the T&Cs and deliberately sharing information publicly covers the GDPR's "consent" lawful base. Even under GDPR this is not a situation where someone signed up for something else and then happen to have their personal data shared as a byproduct. They signed up to a site, agreed to T&Cs, and then explicitly and deliberately shared their personal data.

CobrastanJorji 8y ago
The GDPR's super complicated, but I'm pretty sure its Right of Erasure, and specifically Article 7(3), which gives data subjects the right to withdraw consent at any time and the clause “it shall be as easy to withdraw consent as to give it" trumps any ridiculous "irrevocable" license to distribute your content in any form forever. Also importantly, the GDPR requires that a controller not make a service conditional upon consent. Hacker News is likely not in compliance unless they make such data processing optional and require anyone interested to explicitly opt in. But, then again, I'm not a lawyer, and even if I were, actual lawyers don't seen to know what the hell the GDPR actually requires either.
- dasil003 8y ago
  IAANAL, and I don't mean to single you out here, but this seemingly rational argument strikes me as subtle FUD. It's the type of argument that someone with a vested interest in collecting user data for profit might put forth in the hopes of polarizing the tech community and painting GDPR as out of touch with technical common sense. Again, I'm not accusing you of anything here, I'm just pointing out who benefits from framing the conversation this way. So far there is a lot of precedent for small operators shutting down their sites out of fear of GDPR, but there is actually no precedent for regulators having actually gone after small operators for anything resembling reasonable practices. The day may come where EU regulators try to crack down on forums for who are unwilling or unable to redact users messages post-facto, but we're nowhere close to that today and I don't see strong reason to believe that's where we're headed either.