5 ms·
How hard is it to export all of the comments someone has written on HN? I have been thinking about converting my HN comment history into something like a blog.
by ilaksh 2y ago
How hard is it to export all of the comments someone has written on HN? I have been thinking about converting my HN comment history into something like a blog. Each entry would show the title of the submission, link if applicable, and the comment and link to comment thread.
Maybe you could provide that service for a small fee.
https://github.com/runvnc/hncomments https://github.com/runvnc/hncomments
- deleted 2y ago[deleted]
- ttepasse 2y agoWrite an GDPR request … and wait.
- matsemann 2y agoAlso would be interesting to see my downvotes to upvotes ratio, compared with others.
- mikeqq2024 2y agoIs there notification mechanism for hn when someone comments after you or @you?
- koolala 2y agotoo hard? too easy? i dont know?
- opjjf 2y agoThe Hacker News BigQuery data makes this quite easy: select * from `bigquery-public-data.hacker_news.full` where `by` = 'ilaksh' and type = 'comment' order by timestamp desc EDIT: seems this is out of date
- hipadev23 2y agoClickhouse has an updated dataset: https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPTSBoYWNrZXJuZXdzX2hpc3RvcnkgT1JERVIgQlkgdGltZSBERVNDIExJTUlUIDEwMA== https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT...
- SushiHippie 2y agoOr using the algolia API like so: https://news.ycombinator.com/item?id=40634899 https://news.ycombinator.com/item?id=40634899
- notpushkin 2y agoNot that hard: https://news.ycombinator.com/threads?id=ilaksh https://news.ycombinator.com/threads?id=ilaksh
- fragmede 2y agoYeah but that's paginated by id.
- SushiHippie 2y agoUsing the algolia api https://hn.algolia.com/api/v1/search?tags=author_ilaksh,comment&hitsPerPage=50 https://hn.algolia.com/api/v1/search?tags=author_ilaksh,comm... You'll need to paginate it, as it's limited to 50 results per page, like so: https://hn.algolia.com/api/v1/search?tags=author_ilaksh,comment&hitsPerPage=50&page=1
- krapp 2y agoOr HN's actual API, which isn't rate limited: https://github.com/HackerNews/API https://github.com/HackerNews/API
- SushiHippie 2y agoYes, but with the algolia API, you'll need to make way fewer requests, as you'll get up to 50 comments per API call + you also get the title and URL of the post where this comment was posted in the same call.
- arp242 2y agoI wrote a simple program a while ago to just download all of https://news.ycombinator.com/threads?id=arp242 https://news.ycombinator.com/threads?id=arp242, with my cookie set. Upshot of this is that it will include scores and flagged comments, which public sources won't have. It's useful to filter comments. To be honest I don't remember what the exact status is; it should work because I have a large TOML file with all my comments. But I don't recall if there's anything "TODO" or if I just forgot to publish it. Need to put your cookie in the variable at the top. https://gist.github.com/arp242/4f88069cdc8166d21aa26daac7ffe684 https://gist.github.com/arp242/4f88069cdc8166d21aa26daac7ffe...
- HeatrayEnjoyer 2y agoHow do you access the flagged comments?
- arp242 2y agoIt's just listed under your "threads", like any other comment: https://news.ycombinator.com/threads?id=HeatrayEnjoyer https://news.ycombinator.com/threads?id=HeatrayEnjoyer
- thekingshorses 2y agohttps://hn.premii.com/#/profile/ilaksh/comments https://hn.premii.com/#/profile/ilaksh/comments HN has API.
- crazygringo 2y agoGenuine question, how is that any different from just your list of comments on HN? What you're describing seems pretty identical to: https://news.ycombinator.com/threads?id=ilaksh https://news.ycombinator.com/threads?id=ilaksh Is it just that you want to host it on your own domain?
- GeoAtreides 2y ago> export all of the comments someone has written Please remember that users gave a license to their content only to HN, not everyone. Using their data without their consent might come afoul of copyright laws.
- krapp 2y agoThere are already numerous third party sites that crawl HN and use its API. Your comments here are public, no one needs your permission to use them.
- GeoAtreides 2y agoSo if you put a photo up on instagram I can yoink it and use it as I see fit? I suggest reading the legal print: https://www.ycombinator.com/legal/ https://www.ycombinator.com/legal/ > By uploading any User Content you hereby grant and will grant Y Combinator and its affiliated companies a nonexclusive, worldwide, royalty free, fully paid up, transferable, sublicensable, perpetual, irrevocable license to copy, display, upload, perform, distribute, store, modify and otherwise use your User Content for any Y Combinator-related purpose in any form, medium or technology now known or later developed. It says here I grant Y combinator (and its affiliated companies; hm...) a licence, not that I grant it to some rando with a scrapper (or using an API). Anyone that's not Y combinator (and its affiliated companies; hm....) that's using my content posted here is doing so without any licence to do so from me, potentially violating my copyright rights.
- krapp 2y ago>So if you put a photo up on instagram I can yoink it and use it as I see fit? We're talking about Hacker News, not Instagram. >It says here I grant Y combinator (and its affiliated companies; hm...) a licence, not that I grant it to some rando with a scrapper (or using an API). And yet Hacker News has an API and plenty of people use it, implying that Hacker News considers the use of its own API to be valid within the context of its "license" and by extension any application also using the API. Feel free to lawyer up and sue, though, and let us know how it goes. I don't think it'll go as well as you seem to believe.