5 ms·

Great idea. - The frontpage should directly show the list of papers, like with HN. You shouldn't have to click on "trending" first. (When you are logged in, yo

Great idea. - The frontpage should directly show the list of papers, like with HN. You shouldn't have to click on "trending" first. (When you are logged in, you see a list of featured papers on the homepage, which isn't as engaging as the "trending" page. Again, compare HN: Same homepage whether you're logged in or not.) - Ranking shouldn't be based on comment activity, which ranks controversial papers, rather papers should be voted on like comments. - It's slightly confusing that usernames allow spaces. It will also make it harder to implement some kind of @ functionality in the comments. - Use HTML rather then PDF. Something that could be trivial with HTML, like clicking on an image to show a bigger version, requires you to awkwardly zoom in with PDF. With HTML, you would also have one column, which would fit better with the split paper/comments view.

sestep 2y ago
Tiny note: Stack Exchange also allows spaces in display names, and they make @ functionality work regardless: https://meta.stackexchange.com/a/43020/297476 https://meta.stackexchange.com/a/43020/297476 Agreed that it makes it more complicated though.
throw_pm23 2y ago
Counterpoint: please don't do any of the above and keep arxiv as it is. It is too valuable to mess it up, it is the few things on the internet that have not been ruined yet, and the "comment activity" can happen in the articles themselves at the scale of years, decades, and centuries.
- Epa095 2y ago
  This seems to be a completely different team than arxiv, making a discussion forum on the side. And I prefer this over discussions on 'X'.
- godelski 2y ago
  I actually want the above, BUT I don't want it to be open to everyone. Not all gate keeping is bad. We don't lack places that the public can engage with researchers and experts. What we do lack are places where researchers/experts can communicate with one another __and expect the other person to be a peer__. The bar to arxiv is (absurdly) low, and I think that's fine. Not everything has to be for everyone. My longer comment: https://news.ycombinator.com/item?id=41484123 https://news.ycombinator.com/item?id=41484123 I'm going to go crazy if I get more GitHub issues asking where the source code is or how to fine tune a model. My research project page is not a Google Search engine nor ChatGPT...
- musicale 2y ago
  > it is the few things on the internet that have not been ruined yet Hmm, this is an interesting point.
diggan 2y ago
> - Ranking shouldn't be based on comment activity, which ranks controversial papers, rather papers should be voted on like comments. How about not ranking things at all? I don't feel like things like this should be a popularity/"like" contest and instead let the content of the paper/comments speak for themselves. Yes, there will be some chaff to sort through when reading, but humanity will manage. Just sort things by updated/created/timestamp and all the content will be equal.
- thornewolf 2y ago
  thats ranking by recency, which means i can abuse it by churning low quality content out to arXive
- pessimizer 2y ago
  > let the content of the paper/comments speak for themselves. People can't read everything, and have rely on others to filter up the good stuff. If you read something random, based on no recommendation, it's charity work (the odds are extremely good that it is bad) and you should recommend that thing to other people if it turns out to be useful. Ultimately, that's the entire point of any of this design: if we don't care about any of the metadata on the papers, they could just be numbered text files at an ftp site. The fewer things I have to read to find out they're shit, the longer life I have. I say the opposite: put a lot of thought into how papers are organized and categorized, how comments on papers are organized and categorized, the means through which papers can be suggested to users who may be interested in them, and the methods by which users can inject their opinions and comment into those processes. Figure out how to thwart ways this process can be gamed. Treat the content equally, don't force the content to be equal. Hacker News shouldn't just be the unfiltered new page.
- gus_massa 2y ago
  Sorted by "new"... Most articles are not interesting, most of the interesting ones are interesting only for a niche of a few researchers. The front page will be flowed by uninteresting stuff.
Retr0id 2y ago
> rather papers should be voted on like comments. I don't think this is an inherently better approach, but maybe there should be an option for different ranking mechanisms. You could also rank by things like cite-frequency, cite-recency, "cite pagerank", etc.
- throwthrowuknow 2y ago
  Agree, don’t sink a bunch of effort into creating a ranking algorithm. Expose metrics that users can sort or filter by which will work for both signed in and signed out. If you want to add more tools for signed in users then let them define their own filters that they can save like comment activity plus weighted by author, commenter, recency, topic etc. See the nntp discussion that was on here the other day.
- dartos 2y ago
  Yep. User driven ranking leads to people gaming the system for internet points.
- anamexis 2y ago
  It doesn't seem like citations would be good for discovery, because there must be a significant latency between when a paper is released and when citations start coming in.
  - bee_rider 2y ago
    Probably it would be best to just get a site on the web and expose a bunch of different metrics so people can sort by whatever. Citations are probably not the best metric for discovery, but also this really just makes me wonder if papers are not the best thing for discovery. An academic produces ideas, not papers, those are just a side-effect. The path is something like: * make a idea * write short conference papers about it * present it in conferences * write journal papers about it * maybe somebody writes a thesis about it (Talking to people about it throughout). If we want to discover ideas as they are being worked on, I guess we’d want some proxy that captures whether all that stuff is progressing, and if anybody has noticed… Finding that proxy seems incredibly difficult, maybe impossible.
impendia 2y ago
> Use HTML rather then PDF. The PDF is the original paper, as it appears on arXiv, so using PDF is natural. In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about. That said, the arXiv is rolling out an experimental LaTeX-to-HTML converter for those who prefer HTML, for those who usually prefer PDF but may be just browsing on their phone at the time, or for those who have accessibility issues with PDFs. I just checked this out for one of my own papers; it is not perfect, but it is pretty good, especially given that I did absolutely nothing to ensure that our work would look good in this format: https://arxiv.org/html/2404.00541v1 https://arxiv.org/html/2404.00541v1 So it looks like we're converging towards having the best of both worlds.
- ethanol-brain 2y ago
  > That said, the arXiv is rolling out an experimental LaTeX-to-HTML Some history: https://www.arxiv-vanity.com/ https://www.arxiv-vanity.com/
- throw10920 2y ago
  > In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. The tooling producing PDF by default absolutely makes the preference for PDF justifiable. However, tooling is driven by usage - if more papers come with rendered HTML (e.g. through Pandoc if necessary), and people start preferring to consume HTML, then tooling support for HTML will improve. > But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about. Couldn't you replace references like "the bottom of page 7" with others like "two sentences after theorem 1.2" that are layout-independent? This would also make it easier to rewrite parts of the paper without having to go back and fix all of your layout-dependent references when the layout shifts. HTML has strong advantages for both paper and electronic reading, so I think it's worth making an effort to adopt. When I print out a paper to take notes, the margins are usually too narrow for my note-taking, and I additionally have a preference for a narrow margin on one side and a wide margin on the other (on the same side, not alternating with page parity like a book), which virtually no paper has in its PDF representation. When I read a paper electronically, I want to eliminate pagination and read the entire thing as a single long page. Both of these things are significantly easier to do with HTML than LaTeX (and, in the case of the "eliminate pagination" case, I've never found a way to do it with LaTeX at all). (also, in general, HTML is just far more flexible and accessible than PDF for most people to modify to suit their preferences - I think most on HN would agree with that)
gradus_ad 2y ago
> Ranking shouldn't be based on comment activity, which ranks controversial papers But don't we want people's attention drawn to controversial/conversation generating papers? The whole point of the platform is to drive conversation
- woodson 2y ago
  The concern may be about what effect this will have on future papers (just like news headlines engineered for clickbait).
- godelski 2y ago
  My guess is that they're trying to deal with a problem they're creating: being open to everyone. The problem with places like Twitter, Reddit, Github, HN, etc is that you don't know you're talking to a peer and the idiots asking irrelevant questions or proposing dumb things outnumber researchers a million to one. Even allowing public to upvote or affect the rankings is not beneficial to science. I'm all for casting wide nets and making things available to everyone, but a little gate keeping is not bad (just don't gate keep by race, class, or those things). But I'm sorry, research is hard. There's a reason people spend decades researching things that at face value look trivial. Rabbit holes are everywhere and just because you don't know about them doesn't mean your opinion has equal weight. We seriously lack areas where experts can talk to other experts.
runningmike 2y ago
Many people on earth have names with spaces. So good that a username can reflect a real name a person has.
rehaanahmad 2y ago
Great idea, we'll look into making the home page the trending page soon. Regarding HTMl, our original site actually only supported HTML (because it was easier to build an annotator for an HTML page). the issue is that a good ~25% of these papers don't render properly which pisses off a lot of academics. Academics spend a lot of time making their papers look nice for PDF, so when someone comes along and refactors their entire paper in HTML, not everyone is a fan. That being said, I do think long term HTML makes a lot of sense for papers. It allows researchers to embed videos and other content (think, robotics papers!). At some point we do want to incorporate HTML papers back into the site (perhaps as a toggle).
- DoctorOetker 2y ago
  I apologize for changing topic here: Did you bulk download the arxiv metadata, PDF and or LaTeX files? I am trying to figure out what the required space is for just the most recent version of the PDF's. I can find mentions of the total size in their S3 bucket but unclear if that also includes older versions of the PDF's. I also wonder if the Kaggle dataset is kept up to date since it states merely 1.7M articles instead of 2.4 I read elsewhere. Edit: I just found the answers to my question here: https://info.arxiv.org/help/bulk_data_s3.html https://info.arxiv.org/help/bulk_data_s3.html
ZeroSolstice 2y ago
> The frontpage should directly show the list of papers, like with HN. I disagree. There are numerous times where I have browsed the comments on a HN post where people haven't read the article and are just responding to the comment thread. The workflow for this seems a bit different in that a person would have already read a paper and wanted to read through existing discussions or respond to discussion. With that, having the search front and center would follow as the next steps for a person who read a paper and wanted to "search" for discussions related to that paper in particular. HN is more an aimless browsing which is a bit different than researching a specific area or topic.

sestep 2y ago

Tiny note: Stack Exchange also allows spaces in display names, and they make @ functionality work regardless: https://meta.stackexchange.com/a/43020/297476 https://meta.stackexchange.com/a/43020/297476 Agreed that it makes it more complicated though.

throw_pm23 2y ago

Counterpoint: please don't do any of the above and keep arxiv as it is. It is too valuable to mess it up, it is the few things on the internet that have not been ruined yet, and the "comment activity" can happen in the articles themselves at the scale of years, decades, and centuries.

Epa095 2y ago
This seems to be a completely different team than arxiv, making a discussion forum on the side. And I prefer this over discussions on 'X'.
godelski 2y ago
I actually want the above, BUT I don't want it to be open to everyone. Not all gate keeping is bad. We don't lack places that the public can engage with researchers and experts. What we do lack are places where researchers/experts can communicate with one another __and expect the other person to be a peer__. The bar to arxiv is (absurdly) low, and I think that's fine. Not everything has to be for everyone. My longer comment: https://news.ycombinator.com/item?id=41484123 https://news.ycombinator.com/item?id=41484123 I'm going to go crazy if I get more GitHub issues asking where the source code is or how to fine tune a model. My research project page is not a Google Search engine nor ChatGPT...
musicale 2y ago
> it is the few things on the internet that have not been ruined yet Hmm, this is an interesting point.

diggan 2y ago

> - Ranking shouldn't be based on comment activity, which ranks controversial papers, rather papers should be voted on like comments. How about not ranking things at all? I don't feel like things like this should be a popularity/"like" contest and instead let the content of the paper/comments speak for themselves. Yes, there will be some chaff to sort through when reading, but humanity will manage. Just sort things by updated/created/timestamp and all the content will be equal.

thornewolf 2y ago
thats ranking by recency, which means i can abuse it by churning low quality content out to arXive
pessimizer 2y ago
> let the content of the paper/comments speak for themselves. People can't read everything, and have rely on others to filter up the good stuff. If you read something random, based on no recommendation, it's charity work (the odds are extremely good that it is bad) and you should recommend that thing to other people if it turns out to be useful. Ultimately, that's the entire point of any of this design: if we don't care about any of the metadata on the papers, they could just be numbered text files at an ftp site. The fewer things I have to read to find out they're shit, the longer life I have. I say the opposite: put a lot of thought into how papers are organized and categorized, how comments on papers are organized and categorized, the means through which papers can be suggested to users who may be interested in them, and the methods by which users can inject their opinions and comment into those processes. Figure out how to thwart ways this process can be gamed. Treat the content equally, don't force the content to be equal. Hacker News shouldn't just be the unfiltered new page.
gus_massa 2y ago
Sorted by "new"... Most articles are not interesting, most of the interesting ones are interesting only for a niche of a few researchers. The front page will be flowed by uninteresting stuff.

Retr0id 2y ago

> rather papers should be voted on like comments. I don't think this is an inherently better approach, but maybe there should be an option for different ranking mechanisms. You could also rank by things like cite-frequency, cite-recency, "cite pagerank", etc.

throwthrowuknow 2y ago
Agree, don’t sink a bunch of effort into creating a ranking algorithm. Expose metrics that users can sort or filter by which will work for both signed in and signed out. If you want to add more tools for signed in users then let them define their own filters that they can save like comment activity plus weighted by author, commenter, recency, topic etc. See the nntp discussion that was on here the other day.
dartos 2y ago
Yep. User driven ranking leads to people gaming the system for internet points.
anamexis 2y ago
It doesn't seem like citations would be good for discovery, because there must be a significant latency between when a paper is released and when citations start coming in.
- bee_rider 2y ago
  Probably it would be best to just get a site on the web and expose a bunch of different metrics so people can sort by whatever. Citations are probably not the best metric for discovery, but also this really just makes me wonder if papers are not the best thing for discovery. An academic produces ideas, not papers, those are just a side-effect. The path is something like: * make a idea * write short conference papers about it * present it in conferences * write journal papers about it * maybe somebody writes a thesis about it (Talking to people about it throughout). If we want to discover ideas as they are being worked on, I guess we’d want some proxy that captures whether all that stuff is progressing, and if anybody has noticed… Finding that proxy seems incredibly difficult, maybe impossible.

impendia 2y ago

> Use HTML rather then PDF. The PDF is the original paper, as it appears on arXiv, so using PDF is natural. In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about. That said, the arXiv is rolling out an experimental LaTeX-to-HTML converter for those who prefer HTML, for those who usually prefer PDF but may be just browsing on their phone at the time, or for those who have accessibility issues with PDFs. I just checked this out for one of my own papers; it is not perfect, but it is pretty good, especially given that I did absolutely nothing to ensure that our work would look good in this format: https://arxiv.org/html/2404.00541v1 https://arxiv.org/html/2404.00541v1 So it looks like we're converging towards having the best of both worlds.

ethanol-brain 2y ago
> That said, the arXiv is rolling out an experimental LaTeX-to-HTML Some history: https://www.arxiv-vanity.com/ https://www.arxiv-vanity.com/
throw10920 2y ago
> In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. The tooling producing PDF by default absolutely makes the preference for PDF justifiable. However, tooling is driven by usage - if more papers come with rendered HTML (e.g. through Pandoc if necessary), and people start preferring to consume HTML, then tooling support for HTML will improve. > But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about. Couldn't you replace references like "the bottom of page 7" with others like "two sentences after theorem 1.2" that are layout-independent? This would also make it easier to rewrite parts of the paper without having to go back and fix all of your layout-dependent references when the layout shifts. HTML has strong advantages for both paper and electronic reading, so I think it's worth making an effort to adopt. When I print out a paper to take notes, the margins are usually too narrow for my note-taking, and I additionally have a preference for a narrow margin on one side and a wide margin on the other (on the same side, not alternating with page parity like a book), which virtually no paper has in its PDF representation. When I read a paper electronically, I want to eliminate pagination and read the entire thing as a single long page. Both of these things are significantly easier to do with HTML than LaTeX (and, in the case of the "eliminate pagination" case, I've never found a way to do it with LaTeX at all). (also, in general, HTML is just far more flexible and accessible than PDF for most people to modify to suit their preferences - I think most on HN would agree with that)

gradus_ad 2y ago

> Ranking shouldn't be based on comment activity, which ranks controversial papers But don't we want people's attention drawn to controversial/conversation generating papers? The whole point of the platform is to drive conversation

woodson 2y ago
The concern may be about what effect this will have on future papers (just like news headlines engineered for clickbait).
godelski 2y ago
My guess is that they're trying to deal with a problem they're creating: being open to everyone. The problem with places like Twitter, Reddit, Github, HN, etc is that you don't know you're talking to a peer and the idiots asking irrelevant questions or proposing dumb things outnumber researchers a million to one. Even allowing public to upvote or affect the rankings is not beneficial to science. I'm all for casting wide nets and making things available to everyone, but a little gate keeping is not bad (just don't gate keep by race, class, or those things). But I'm sorry, research is hard. There's a reason people spend decades researching things that at face value look trivial. Rabbit holes are everywhere and just because you don't know about them doesn't mean your opinion has equal weight. We seriously lack areas where experts can talk to other experts.

runningmike 2y ago

Many people on earth have names with spaces. So good that a username can reflect a real name a person has.

rehaanahmad 2y ago

Great idea, we'll look into making the home page the trending page soon. Regarding HTMl, our original site actually only supported HTML (because it was easier to build an annotator for an HTML page). the issue is that a good ~25% of these papers don't render properly which pisses off a lot of academics. Academics spend a lot of time making their papers look nice for PDF, so when someone comes along and refactors their entire paper in HTML, not everyone is a fan. That being said, I do think long term HTML makes a lot of sense for papers. It allows researchers to embed videos and other content (think, robotics papers!). At some point we do want to incorporate HTML papers back into the site (perhaps as a toggle).

DoctorOetker 2y ago
I apologize for changing topic here: Did you bulk download the arxiv metadata, PDF and or LaTeX files? I am trying to figure out what the required space is for just the most recent version of the PDF's. I can find mentions of the total size in their S3 bucket but unclear if that also includes older versions of the PDF's. I also wonder if the Kaggle dataset is kept up to date since it states merely 1.7M articles instead of 2.4 I read elsewhere. Edit: I just found the answers to my question here: https://info.arxiv.org/help/bulk_data_s3.html https://info.arxiv.org/help/bulk_data_s3.html

ZeroSolstice 2y ago

> The frontpage should directly show the list of papers, like with HN. I disagree. There are numerous times where I have browsed the comments on a HN post where people haven't read the article and are just responding to the comment thread. The workflow for this seems a bit different in that a person would have already read a paper and wanted to read through existing discussions or respond to discussion. With that, having the search front and center would follow as the next steps for a person who read a paper and wanted to "search" for discussions related to that paper in particular. HN is more an aimless browsing which is a bit different than researching a specific area or topic.