7 ms·
Ask HN: Can I download my HN data?
Github recently announced the ability to download all your Github data[0]. Facebook, Twitter, Instagram, and Google all offer similar services. Is there a way to do this for my comments, votes, and submissions on HackerNews?
[0] https://blog.github.com/2018-12-19-download-your-data/
- hoppelhase 8y agoThe stuff you can't scrape is the interesting stuff.
- sgillen 8y agoWhat stuff can you not scrape?
- 1337biz 8y agoHow somebody voted/flagged on posts.
- hoppelhase 8y agoSessions, IP addresses and User Angents associated with oneself.
- sigjuice 8y agoDo we even know whether HN stores this information?
- hoppelhase 8y agoWell, no.
- deleted 8y ago[deleted]
- activekerrar 8y agoNot directly, but you may find their API useful: https://github.com/HackerNews/API https://github.com/HackerNews/API
- megous 8y agoScrape it? It's in your profile.
- beokop 8y agoIf you’re in the EU you can refer to the GDPR and request all the data they have on you.
- yorwba 8y agoSure you can, but HN is not GDPR-compliant, so you'll just get pointed at the public API.
- twtw 8y agoHow can HN just choose to be noncompliant? Aren't there penalties? I don't understand the jurisdiction of GDPR very well, but I thought it applied to all EU users.
- beokop 8y agoIt does apply to all EU users and they risk fines if they don’t comply.
- ydnaclementine 8y agoA law is only as good as it is enforced
- mattr47 8y agoIt does not apply as HN is not in the EU. If an EU citizen does not want their data collected then they can choose not to participate, as the EU has no jurisdiction over HN.
- mdekkers 8y agoThat isn't how the GDPR works. There are many GDPR primers on the web, here is a random one. https://www.recode.net/2018/5/16/17360944/gdpr-us-business-eu-european-union-data-protection-privacy https://www.recode.net/2018/5/16/17360944/gdpr-us-business-e...
- tlrobinson 8y agoFYI it's from 2015 but there's a dump in the BigQuery public datasets: https://bigquery.cloud.google.com/dataset/fh-bigquery:hackernews?pli=1 https://bigquery.cloud.google.com/dataset/fh-bigquery:hacker... (along with a bunch of other interesting things)
- jaredsohn 8y agoAs others say, you can get the comments and submissions via the API. To start with you can try this for the public information: http://hnuser.herokuapp.com/user/conroy/json http://hnuser.herokuapp.com/user/conroy/json I wrote hnuser (http://hnuser.herokuapp.com http://hnuser.herokuapp.com) a long time ago to show karma over time and haven't looked at in awhile but at first glance it seems to still work. It also exists as an npm package if you want to run it from the commandline.
- nicolashahn 8y agoI wrote a small python library to assist in querying the HN search API a little while back: https://github.com/nicolashahn/py-search-hn https://github.com/nicolashahn/py-search-hn I also have an example of how to get all of a single user's comments: https://github.com/nicolashahn/py-search-hn/blob/master/get_user_comments.py https://github.com/nicolashahn/py-search-hn/blob/master/get_... You could use a similar method for submissions. I'm not sure how getting all the stories or comments you've voted on would work, however.
- StephenMarley 8y agoPeople find it hard to stay committed again. It’s becoming a thing. Getting information & data you need is quite not a big deal. Sometimes the truth needs to be unveiled by whatsoever means necessary.The latter of the case should always be reckoned with, of which it would be known eventually what would be the data at hand afterwards. definitely(BOBHACKER911•(<@>)• GMA;|) would do justice on this intercepting with wares. All will be well.