5 ms·
We were frustrated by the experience of reading machine learning papers on screens (particularly phones/tablets). There are lots of good tools for authoring HTM
by bfirsh 9y ago
We were frustrated by the experience of reading machine learning papers on screens (particularly phones/tablets). There are lots of good tools for authoring HTML papers (Distill, Authorea, etc) but nothing that deals with the vast number of PDF papers that already exist.
So, we built Arxiv Vanity: a site that renders Arxiv papers as web pages. It’s still pretty janky, but for the papers that do render correctly, the experience is so much better than reading a PDF. For example:
https://www.arxiv-vanity.com/papers/1705.04085v3/ https://www.arxiv-vanity.com/papers/1705.04085v3/
https://www.arxiv-vanity.com/papers/1708.00884/ https://www.arxiv-vanity.com/papers/1708.00884/
https://www.arxiv-vanity.com/papers/1705.06031v2/ https://www.arxiv-vanity.com/papers/1705.06031v2/
The source for the LaTeX to HTML renderer is on GitHub[0]. It’s built on Pandoc[1] and Distill.pub’s template[2].
[0] https://github.com/arxiv-vanity/engrafo https://github.com/arxiv-vanity/engrafo
[1] https://pandoc.org https://pandoc.org
[2] https://github.com/distillpub/template https://github.com/distillpub/template
- Drup 9y agoI was looking for a way to turn my (soon-to-be-defended) PhD thesis into an epub, and investigated the various LaTeX2Html converters. I was pretty disappointed when I realized that all of them are terrible and have no hope of handling my manuscript. My current solution is to create a rendering of my thesis in a5 format. :/ This look quite a bit better, so here is the question: what do you not support at the moment?
- bfirsh 9y agoA lot of things. LaTeX and its packages have so much surface area. Our approach so far is to just make the papers that we read readable. That probably covers the 20% of LaTeX features that 80% of people use. Here is the broken stuff we are keeping track of: https://github.com/arxiv-vanity/engrafo/issues https://github.com/arxiv-vanity/engrafo/issues (feel free to add to it!)
- zzleeper 9y agoIs there a reason for relying on pandocfilters instead of on Panflute [1]? I would think that panflute would allow for more readable code, which helps whendealing with all the corner cases and rough edges of latex. [1] https://github.com/sergiocorreia/panflute https://github.com/sergiocorreia/panflute
- bfirsh 9y agoBecause we didn't know that existed! That looks so much better, thank you. The pandocfilters library is really hard to use. https://github.com/arxiv-vanity/engrafo/issues/160 https://github.com/arxiv-vanity/engrafo/issues/160
- KGIII 9y agoHave you seen pandoc? That should be able to do that and the comments about results are usually positive. https://pandoc.org/epub.html https://pandoc.org/epub.html One final thing, and wildly off topic, is that when you do your defense, remember that you probably know more about the specifics of the subject than anyone else in the room. Many folks stress over it, but you're almost certainly going to be the actual expert in the room. Good luck!
- deleted 9y ago[deleted]
- nextos 9y agoIt's very nice. You should expand to cover bioRxiv (biology) too.
- aroch 9y agobioRxiv doesn't expose LateX files, they explicitly only use PDFs to make things easier. Which means you're going to need to reflow PDFs (a la https://docushow.com/ https://docushow.com/), and I would guess there are a lot more edge cases there
- ldenoue 9y agoThanks for linking to https://docushow.com https://docushow.com Also a work in progress, but PDF reflow is a hard problem so you never ship if you want to solve all cases :) Your solution using the LaTex source generates really nice HTML, congrats!
- zellyn 9y agoLovely idea, and I can't wait until it gains super deep-learning smarts and gets everything perfect :-) For now, it's hard to read: https://docushow.com/viewdoc?url=https%3A%2F%2Farxiv.org%2Fpdf%2F1710.07600v1.pdf https://docushow.com/viewdoc?url=https%3A%2F%2Farxiv.org%2Fp...
- leephillips 9y agoIn all three cases I find the original PDFs more pleasant to read. HTML typography is not up to snuff. I read them on a laptop, however, and I can see that this would be useful if one is forced to read on a phone. (One thing that is very ugly in the PDFs, and most scholarly papers, is the use of different-colored boxes for hyperlinks. Authors, please consider putting \usepackage[colorlinks]{hyperref} in your LaTeX preambles.)
- Cyph0n 9y agoJust tried your suggestion: it ends up looking much uglier with font colors imo.
- leephillips 9y agoThe default saturated colors are a bit garish. But you can set them to be anything you want. See the hyperref documentation.
- sp332 9y agoFor me the PDF was fuzzier https://imgur.com/a/WJ5y3 https://imgur.com/a/WJ5y3 and the HTML version was more convenient to read in a single column. The two-column format is nice if I'm skimming to see if a paper is going to be interesting, but when I sit down to read it the HTML version definitely wins.
- JBorrow 9y agoOne of the things that I came across when writing my own janky pdf/latex->html converter for lecture notes[0] is that Pandoc doesn't handle references and subfigures correctly, even with pandoc-crossref and pandoc-citeproc enabled. I had to write a little python module[1] that used regex to extract those and then handle them on my own separately... This is definitely something you should look at. [0] https://dmaitre.phyip3.dur.ac.uk/NPP/notes/ https://dmaitre.phyip3.dur.ac.uk/NPP/notes/ [1] https://github.com/JBorrow/latex-pandoc-preprocessor https://github.com/JBorrow/latex-pandoc-preprocessor
- Eridrus 9y agoThis is cool, it would be nice to have a chrome extension to take me directly to this from the page/pdf.
- bfirsh 9y agoYes! This is a great idea, and something we have been thinking of. https://github.com/arxiv-vanity/arxiv-vanity/issues/67 https://github.com/arxiv-vanity/arxiv-vanity/issues/67
- isaykatsman 9y agoI actually made one quickly, published here: https://chrome.google.com/webstore/detail/arxiv-vanity-plugin/jfnlkegibnoaagfdabjkchhocdhnoofk https://chrome.google.com/webstore/detail/arxiv-vanity-plugi... . It injects the arxiv vanity link on abstract pages and if you click the button when viewing an online arxiv pdf it opens the respective arxiv vanity link.
- KGIII 9y agoI'm not sure I understand. Well, I understand what you're doing but I'm not sure why you'd dislike PDF. PDF has the great benefit of rendering the same on every system. With very few exceptions, PDF will look exactly the same on every system and will print the same on every system. HTML doesn't really have that same benefit. Don't get me wrong, I think your service is a great idea for those who would like HTML formatted results, but I'm not understanding the complaint about PDF. Could you expand on why you don't like PDF?
- gnicholas 9y agoNot the OP, but PDF is bad on tablets and horrible on phones.
- deleted 9y ago[deleted]
- Cacti 9y agoWhat size tablets are people talking about? I find the iPad size practically ideal for consuming PDF papers.
- grzm 9y agoI think part of the issue is that PDF isn't responsive to the size of the device. A PDF is not much more than an image from the perspective of layout. I'd love to be able to reflow text from a PDF such that a single column fills my screen edge-to-edge and scrolling allows me to advance through the paper, as opposed to requiring me to reposition the viewport every time I reach the end of a column. I know this isn't the purpose of PDFs, and I love them in different contexts where layout (including typography) does matter to me. But I also really want to be able to easily consume papers in a way that isn't constrained by the PDF layout. Yes, I want my cake and a pony. Cakepony.
- KGIII 9y agoYup. I agree. I even find the experience fantastic on the original iPad, as well as a brand new one. I find it just fine on my phone, which isn't nearly as large a screen. I am guessing it is an individual taste thing. That makes some sense.
- Myrmornis 9y ago> particularly phones/tablets I understand the problem with a phone, but PDFs on an ipad/tablet are beautiful and a joy to read. Much better to read the text as originally typeset than to put it through a process such as this which risks corrupting minor but important details in the mathematical content. On my phone I put it in landscape mode and that allows me to read a PDF OK, but I don't really get why one would read academic papers on a phone, why not use a tablet? However I'm very interested in engrafo. It sounds like it will allow me to automatically publish blog style content from my LaTeX sources without having to fork the LaTeX content into a markdown / HTML version. I just don't understand why you don't like reading academic papers as PDFs on tablets!
- popcorncolonel 9y agoIt would be amazing if we could browse "Latest" by category, and for a certain day, much like: https://arxiv.org/list/math.NT/recent https://arxiv.org/list/math.NT/recent
- apepe 9y agoThat first article and how it looks when imported into Authorea in one click: https://www.authorea.com/users/3/articles/208068-automatic-extrinsic-calibration-for-lidar-stereo-vehicle-sensor-setups https://www.authorea.com/users/3/articles/208068-automatic-e... (just a couple of labels and si units which do not render). Note: it is forkable and can be commented upon.