7 ms·

Show HN: Top PDFs and Papers Submitted to Hacker News

250 points by cloudkj 8y ago

karmakaze 8y ago
It would be great if these could be tagged into a few categories.
- cloudkj 7y ago
  That sounds like a fun and useful follow up. My initial thoughts would be to do some unsupervised topic modeling with LDA. Happy to consider other suggestions.
  - mathnmusic 7y ago
    Why not add these research papers in learn-awesome which is building a topic graph? https://github.com/learn-awesome/learn-awesome https://github.com/learn-awesome/learn-awesome
Yetanfou 8y ago
Could you add links to the relevant threads on HN to this listing e.g. by linking the HN score there? This would give access to both the PDF for those who just want to read the thing as well as to the discussion for those who wonder what made it reach such dazzling heights on HN.
- kbenson 8y ago
  A good suggestion. For those that want to look up the conversion before that suggestion is (hopefully) implemented, you can use the search of HN to search by submission URLs.
- cloudkj 8y ago
  Thanks for the suggestion. I went ahead and made the change to link the score to the original discussion. It will take a few moments for the CDNs to propagate the changes. One caveat is that some stories have multiple submissions; I just linked to the one with the highest score for now, but will need to iterate a bit to better handle multiple submissions.
epicwhaleburger 8y ago
Related: https://fermatslibrary.com/ https://fermatslibrary.com/
minimaxir 8y ago
Hmm. Reverse-engineering this page with BigQuery is surfacing a lot more results than the page itself (for 2019 atleast). https://docs.google.com/spreadsheets/d/1he6ca0BBYbj2ZEOEpu8L7Rqab8XLJoqBhkXYdSDVreE/edit?usp=sharing https://docs.google.com/spreadsheets/d/1he6ca0BBYbj2ZEOEpu8L... #standardSQL SELECT id, title, url, score FROM `bigquery-public-data.hacker_news.full` WHERE timestamp > '2019-01-01' AND REGEXP_CONTAINS(url, '.pdf|arxiv.org') ORDER BY score desc
- seedie 7y ago
  Thanks to your post I just learned about the big query public datasets https://www.reddit.com/r/bigquery/wiki/datasets https://www.reddit.com/r/bigquery/wiki/datasets
ZoomStop 8y ago
A link at the top for 2019 would be helpful, maybe change text color for year being displayed? When navigating years returning to 2019 isn't intuitive.
- cloudkj 7y ago
  Thanks for the suggestion. I added a link for 2019, and the style for the active year is now bolded. I didn't want to deviate too much from the default color scheme of the theme I'm using, but that may change in the future.
burtonator 8y ago
We did one for Polar at the end of 2018: https://getpolarized.io/2019/01/08/top-pdfs-of-2018-hackernews.html https://getpolarized.io/2019/01/08/top-pdfs-of-2018-hackerne... If you want an awesome PDF reader to read these you should check out Polar :) Yes. There's a Linux version! ;) https://getpolarized.io/ https://getpolarized.io/
baby 8y ago
Now if we could categorize this so that I can get the top crypto :)
sytelus 7y ago
One of the huge issue on HN is lack of good algorithms for ranking. Currently all posts suffer from insufficient initial exposure to receive enough votes and thus false negatives are extremely high on HN. Just go to new page right now, take a look at most recent posting and you will find that majority of good genuine posts would never make it beyond their initial single upvote. Very small subset gets re-posted few times and only randomly some will get attention. Unfortunately papers are probably the biggest victim so you could have as well have collected all the links, did kind-of-random sort and you might still get fairly decent results. The bottom line is that never rely on HN's flawed ranking. Use that as weak signal, combine with Reddit, citations, tweet counts etc if you want to work around vast sea of false negatives that HN currently is.
- ignoramous 7y ago
  Back when pg used to run the community [0], he'd deeply think about problems [1] and then later solve them [2] but not without controversy [3]. news.yc was better for it with such a strong personality at the helm who'd withstand the backlash and hold on to their convictions. I guess, we need someone of that caliber, someone to step up to usher in to the post-pg era [4], as it were. What I do to find my share of interesting links is to search on https://hn.algolia.com https://hn.algolia.com with different keywords and sift through those discussions, follow links in the comments, follow the commentators previous submissions and posts, follow other submissions from the website, gather new keywords and search again... it is kind of a rabbit hole once you start. Though you'd quickly begin to realise that it is great folks continue to submit links that don't end up hitting the front-page, as the aggregation of links is valuable even though the discussion is missing [5]. That said, I have stumbled upon a lot of interesting things aren't simply shared at all on news.yc, though, I guess this is by design. I just hope folks don't stop sharing links just because their previous submissions didn't generate discussion or upvotes on news.yc. [0] https://news.ycombinator.com/item?id=7494708 https://news.ycombinator.com/item?id=7494708 [1] https://news.ycombinator.com/item?id=2403696 https://news.ycombinator.com/item?id=2403696 [2] http://paulgraham.com/hackernews.html http://paulgraham.com/hackernews.html [3] https://news.ycombinator.com/item?id=3122233 https://news.ycombinator.com/item?id=3122233 [4] https://hn.algolia.com/?query=%22Ask%20PG:%22&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=%22Ask%20PG:%22&sort=byPopular... [5] Some would argue that few news.yc comments are worth their weight in upvotes anyway: https://danluu.com/hn-comments/ https://danluu.com/hn-comments/
danielecook 7y ago
I put together something similar but for articles people are linking to on Arxiv here: https://hntrending.com/links/all/arxiv/index.html https://hntrending.com/links/all/arxiv/index.html
- Bolderman 7y ago
  Seems like it hasn't been updated for a few weeks. But it's a cool site!
itdog 7y ago
It will be convenient if there's a search box on the page

250 points by cloudkj 8y ago

karmakaze 8y ago

It would be great if these could be tagged into a few categories.

cloudkj 7y ago
That sounds like a fun and useful follow up. My initial thoughts would be to do some unsupervised topic modeling with LDA. Happy to consider other suggestions.
- mathnmusic 7y ago
  Why not add these research papers in learn-awesome which is building a topic graph? https://github.com/learn-awesome/learn-awesome https://github.com/learn-awesome/learn-awesome

Yetanfou 8y ago

Could you add links to the relevant threads on HN to this listing e.g. by linking the HN score there? This would give access to both the PDF for those who just want to read the thing as well as to the discussion for those who wonder what made it reach such dazzling heights on HN.

kbenson 8y ago
A good suggestion. For those that want to look up the conversion before that suggestion is (hopefully) implemented, you can use the search of HN to search by submission URLs.
cloudkj 8y ago
Thanks for the suggestion. I went ahead and made the change to link the score to the original discussion. It will take a few moments for the CDNs to propagate the changes. One caveat is that some stories have multiple submissions; I just linked to the one with the highest score for now, but will need to iterate a bit to better handle multiple submissions.

epicwhaleburger 8y ago

Related: https://fermatslibrary.com/ https://fermatslibrary.com/

minimaxir 8y ago

Hmm. Reverse-engineering this page with BigQuery is surfacing a lot more results than the page itself (for 2019 atleast). https://docs.google.com/spreadsheets/d/1he6ca0BBYbj2ZEOEpu8L7Rqab8XLJoqBhkXYdSDVreE/edit?usp=sharing https://docs.google.com/spreadsheets/d/1he6ca0BBYbj2ZEOEpu8L... #standardSQL SELECT id, title, url, score FROM `bigquery-public-data.hacker_news.full` WHERE timestamp > '2019-01-01' AND REGEXP_CONTAINS(url, '.pdf|arxiv.org') ORDER BY score desc

seedie 7y ago
Thanks to your post I just learned about the big query public datasets https://www.reddit.com/r/bigquery/wiki/datasets https://www.reddit.com/r/bigquery/wiki/datasets

ZoomStop 8y ago

A link at the top for 2019 would be helpful, maybe change text color for year being displayed? When navigating years returning to 2019 isn't intuitive.

cloudkj 7y ago
Thanks for the suggestion. I added a link for 2019, and the style for the active year is now bolded. I didn't want to deviate too much from the default color scheme of the theme I'm using, but that may change in the future.

burtonator 8y ago

We did one for Polar at the end of 2018: https://getpolarized.io/2019/01/08/top-pdfs-of-2018-hackernews.html https://getpolarized.io/2019/01/08/top-pdfs-of-2018-hackerne... If you want an awesome PDF reader to read these you should check out Polar :) Yes. There's a Linux version! ;) https://getpolarized.io/ https://getpolarized.io/

baby 8y ago

Now if we could categorize this so that I can get the top crypto :)

sytelus 7y ago

One of the huge issue on HN is lack of good algorithms for ranking. Currently all posts suffer from insufficient initial exposure to receive enough votes and thus false negatives are extremely high on HN. Just go to new page right now, take a look at most recent posting and you will find that majority of good genuine posts would never make it beyond their initial single upvote. Very small subset gets re-posted few times and only randomly some will get attention. Unfortunately papers are probably the biggest victim so you could have as well have collected all the links, did kind-of-random sort and you might still get fairly decent results. The bottom line is that never rely on HN's flawed ranking. Use that as weak signal, combine with Reddit, citations, tweet counts etc if you want to work around vast sea of false negatives that HN currently is.

ignoramous 7y ago
Back when pg used to run the community [0], he'd deeply think about problems [1] and then later solve them [2] but not without controversy [3]. news.yc was better for it with such a strong personality at the helm who'd withstand the backlash and hold on to their convictions. I guess, we need someone of that caliber, someone to step up to usher in to the post-pg era [4], as it were. What I do to find my share of interesting links is to search on https://hn.algolia.com https://hn.algolia.com with different keywords and sift through those discussions, follow links in the comments, follow the commentators previous submissions and posts, follow other submissions from the website, gather new keywords and search again... it is kind of a rabbit hole once you start. Though you'd quickly begin to realise that it is great folks continue to submit links that don't end up hitting the front-page, as the aggregation of links is valuable even though the discussion is missing [5]. That said, I have stumbled upon a lot of interesting things aren't simply shared at all on news.yc, though, I guess this is by design. I just hope folks don't stop sharing links just because their previous submissions didn't generate discussion or upvotes on news.yc. [0] https://news.ycombinator.com/item?id=7494708 https://news.ycombinator.com/item?id=7494708 [1] https://news.ycombinator.com/item?id=2403696 https://news.ycombinator.com/item?id=2403696 [2] http://paulgraham.com/hackernews.html http://paulgraham.com/hackernews.html [3] https://news.ycombinator.com/item?id=3122233 https://news.ycombinator.com/item?id=3122233 [4] https://hn.algolia.com/?query=%22Ask%20PG:%22&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=%22Ask%20PG:%22&sort=byPopular... [5] Some would argue that few news.yc comments are worth their weight in upvotes anyway: https://danluu.com/hn-comments/ https://danluu.com/hn-comments/

danielecook 7y ago

I put together something similar but for articles people are linking to on Arxiv here: https://hntrending.com/links/all/arxiv/index.html https://hntrending.com/links/all/arxiv/index.html

Bolderman 7y ago
Seems like it hasn't been updated for a few weeks. But it's a cool site!

itdog 7y ago

It will be convenient if there's a search box on the page