7 ms·

> I expected that everybody would publish latex files (raw source, not PDFs) in computationally accessible formats, with raw data in easily parseable, semantic

> I expected that everybody would publish latex files (raw source, not PDFs) in computationally accessible formats, with raw data in easily parseable, semantic forms. Do you realize that many researchers would be ashamed if others saw their latex code? A friend worked for a journal, he kept a folder of the worst that he had ever seen, and could rant for hours one some gems. Including some papers that were beautifully typeset, about elegant combinatorics/type theory, and were coded on the latex equivalent of a spaceship cobbled with recycled plastic bottles, scraps and duct tape. Do you imagine going through thousand lines macro tex file that grew over the years, when Latex is one of the worst mess ever, with conflicting packages, absurd syntactic 'features', and legacy crippling flaws that make debugging an horrible task? Please, continue to compile your .tex.

soperj 11y ago
It would be much easier to get better if you could view other's latex code.
- pauloday 11y ago
  And if others could contribute improvements on yours.
- anigbrowl 11y ago
  I will never understand why latex devotees think everyone else should be really into typesetting as a hobby. It's like sneering at photographers who don't want to mix darkroom chemicals.
  - swiley 11y ago
    You have obviously never used troff then.
  - p4wnc6 11y ago
    That's not a good comparison, because mixing your own chemicals for photo development adds more time, effort, and defect rate to the process. Becoming more proficient with TeX saves time and makes for more re-usable typesetting structures, which allow you to accurately do what you want faster and faster. It's similar to using emacs or vim. You could say, "Why should programming devotees expect others to adopt those editors?" It's about productivity. I'm a productivity devotee and when I hear people use short-term thinking to discount how valuable the up-front costs for learning TeX (or emacs/vim, etc) are, it motivates me to help them see why it's not true. Of course, I don't want to be dogmatic either. If something works for you and your personal utility function discounts the value of things like TeX or emacs, that's fine.
    - amelius 11y ago
      > It's about productivity. No it is about aesthetics, which are mostly irrelevant for 99% of all research papers.
      - Spivak 11y ago
        It's not just aesthetics, having proper typesetting makes papers more accessible, easier to read and parse, the standard formatting makes understanding the layout of papers easier, there is a huge library of packages that cleanly abstract the task of rendering obscure symbols and complex mathematical formulas, and it automates the job of tracking figure, equation and reference citations to name just a few of the features. Fine, TeX is an ugly language that only a programmer could love, but in over twenty years nobody has yet been able to replace its expressive power and flexibility -- doc generators just compile to LaTeX and more user friendly GUIs like Lyx and TeXMaker have been written to ease the process of writing for end users, but the underlying format has yet to be replaced in over twenty years. Like Markdown, TeX rides the line between being able to be written by hand for greater flexibility and automatically generated for newbies. Unless you think that reading red text on a blue background in payprus would be an equally enjoyable reading experience it more than simple aesthetics.
        deleted 11y ago
        [deleted]
        wonderlust 11y ago
        ... did I just read a rant about how enjoyability and simplicity are NOT aesthetics?
        ninguem2 11y ago
        Are you related to the author of "The Joy of Tex"?
        smsm42 11y ago
        You sound like aesthetics is a bad thing you need to disavow for some reason. > Fine, TeX is an ugly language that only a programmer could love, but in over twenty years nobody has yet been able to replace its expressive power and flexibility Which doesn't make it less ugly. If we can't cure common cold yet, that is no reason to praise it and making it sound as if it's a great thing. It is a problem and so is the need to use ugly language to produce aesthetically pleasing papers. The fact that we don't have solution for this problem yet does not turn it into non-problem. It just says nobody yet was good enough to solve it.
  - mchaver 11y ago
    At the very least it would make sense to provide the source that outputs the document. That could be something like OpenOffice or Latex. There are probably some WYSIWYG solution for Latex. If you are referencing a table from another paper and you want to show or alter it, it would be a lot quicker if you have the code that renders it. If you are in the business of writing papers that you want people to read and quote, it seems like a good idea to master the tools that help you do that.
    - dorfsmay 11y ago
      https://www.lyx.org/ https://www.lyx.org/
      - tormeh 11y ago
        LyX is salvation. Edit in WYSIWYG, export to latex or PDF.
  - cbd1984 11y ago
    Good LaTeX is less about aesthetics and more about having boilerplate code and macros which greatly simplify the process of making readable documents with citations in the correct (nitpicky, archaic, utterly idiosyncratic) format and figures correctly included and labeled and captioned and scaled. The joy of LaTeX is that it handles the aesthetics for you. You can focus entirely on structure. Or that's how it works when you have good boilerplate to start from.
  - gaur 11y ago
    I think the problem is that many people who don't use LaTeX (and some who do) are so inept at typography that it actually impedes the reader's ability to use the document. One LaTeX-agnostic example is when people abuse hyphens-like this-which makes it impossible to parse the sentence without reading it several times.
    - anigbrowl 11y ago
      Well you're onto something there. I liked typography for its own sake long before it became affordable to do it on personal computers, so I suppose I take my competence in that area for granted.
- jessriedel 11y ago
  FYI, the .tex source is available for all papers uploaded to the arXiv.
  - semi-extrinsic 11y ago
    No, you can upload just the PDF if you want.
    - jessriedel 11y ago
      OK, correction: the .tex source is available for almost all papers uploaded to the arXiv (because they give a big warning/plea message to do so).
- JohnHammersley 11y ago
  You can now view (& compile) a large selection of latex examples, templates and articles directly in the browser on platforms such as Overleaf[1]. I'm one of the founders, and we're also making it possible to directly submit projects from Overleaf to repositories such as the arXiv and bioRxiv, and to traditional journals, to help speed up the submission and publication process.[2] [1] https://www.overleaf.com https://www.overleaf.com [2] https://www.overleaf.com/tutorial https://www.overleaf.com/tutorial
mentat 11y ago
Or, if published, we could develop tools to normalize the data and improve the state of the art.
- chejazi 11y ago
  This is the right approach. Like what OCR does for digitizing volumes of literature.
  - dekhn 11y ago
    OCR for historical literature is a really hard problem. Beyond the fact that OCR quality is pretty low, perception of the meaning of the papers isn't addressed by that. So it makes it easier to access (can read papers on the net rather than going to a library) but that's about it.
    - chejazi 11y ago
      > it makes it easier to access (can read papers on the net rather than going to a library) but that's about it. I disagree. The text is indexed and searchable. > the meaning of the papers isn't addressed by that. I agree. I wonder if starting with research papers that have lots of symbolic logic (e.g. math proofs) would be the easiest starting point for a system like this.
jhbadger 11y ago
Also, (experimental) biologists tend not to use TeX/LaTeX. Typically they use Microsoft Word and a proprietary citation manager like Endnote. Typesetting is seen as something the journal deals with, not the author.
- dekhn 11y ago
  I was a biologist, but I used LaTeX: it was important to have parseable references in a flat file.
  - leemailll 11y ago
    Many of biologists stick to word file, especially the review/communet function. It is often because the collaborators stick to word and you have to follow up with it.
    - dekhn 11y ago
      back in the day when I did this, I ended up writing papers with biologists. All the biologists would be cc'd on an email with the doc as an attachment and a "token" was passed between people to determine doc ownership. The person would make their changes and reattach the new doc to the email and pass the token to the next person. Ultimately, this would fall over as people accidentally made concurrent changes and it would be hard if not impossible to merge between two documents. Doc files were never designed to be put into a version control system with merge semantics. Some people use shared folders with locking but locking doesn't work well over file shares. I'm not sure but I suspect now people use dropbox instead of email; it's not clear to me how they manage conflicts and merges. I rather my format be a text-line-based non-rendered format that I can render down if need be, but use the full power of patch and diff to manage conflicts.
agumonkey 11y ago
Shame in 'science', I find that unbelievably absurd. About the 2nd part, may I rephrase it as PDFs are the docker of publication .. (Adobe would probably be very proud of that).
dekhn 11y ago
I wasn't defending LaTeX; I don't use it any more. I am certainly aware that many scientists don't want to publish their code out of embarrassment. It was a surprise to me, but I've heard that feedback. I investigated various alternatives after I found LaTeX too arcane, including DocBook. It's still not entirely clear what solution exists to the collection of problems I have (want to semantically mark up my paper, want it to display nicely regardless of screen, etc).
- eru 11y ago
  What about something like markdown plus some syntax for formulas? You can compile the markdown (via a detour to LaTeX perhaps) to something that looks nice.
  - dekhn 11y ago
    Yeah, some form of enhanced markdown, with self-rendering papers on github (or whatever-hub or whatever-xiv) seems like a very practical solution. It couples the paper's representation to the version control system in a way that could be convenient and expressive.
    - cbd1984 11y ago
      I figure that any markdown dialect enhanced far enough to be usable would be as complex as LaTeX, only without the tooling and pre-written macros.
      - dorfsmay 11y ago
        Enhanced versions of markdown are just turning it back into a full wiki language.
    - dorfsmay 11y ago
      Markdown gets very limited very quickly. Are you aware of pandoc? http://pandoc.org/ http://pandoc.org/
beambot 11y ago
HA! Not to mention unflattering comments in LaTeX or BibTeX files. You thought source code had expletives... yeah.
- vlasev 11y ago
  Some of my .tex files may or may not contain a lot of profanities.
chriswarbo 11y ago
> Do you realize that many researchers would be ashamed if others saw their latex code? I really don't get this argument; I see it come up regarding open data for reproducibility too. TeX/LaTeX source is just an extra piece of metadata which might help someone out in a few years time. That's nothing to feel ashamed of, and it's certainly less shameful than not providing the code. If the code allows flaws in the research to be exposed, that's either something to be proud of, or (if it was intentional) it's not the code that's shameful. My own LaTeX is mostly an inconsistent mixture of copypasta from stackexchange. That doesn't mean I'm ashamed of it, and it's all in public Git repos at http://chriswarbo.net/git http://chriswarbo.net/git if any human or bot actually cares to look. Speaking of which, the commit histories of those Git repositories aren't very good either. Lots of commit messages like "Fixes", or "Oops, typo in last commit", with multiple changes bundled into single commits, atomic changes spread across multiple commits, etc. But, in the same way as my LaTeX source, it's better than having no commit history, which would be the case if I set myself higher standards. Of course, I can follow higher standards, but that's when I'm paid to and/or when collaboration is important. For personal stuff, I'd rather invest the time I save interacting with Git on writing more tests, or refactoring, etc.
slang800 11y ago
> Do you realize that many researchers would be ashamed if others saw their latex code? The exact same think could be said about HTML, but that hasn't stopped people...

soperj 11y ago

It would be much easier to get better if you could view other's latex code.

pauloday 11y ago
And if others could contribute improvements on yours.
anigbrowl 11y ago
I will never understand why latex devotees think everyone else should be really into typesetting as a hobby. It's like sneering at photographers who don't want to mix darkroom chemicals.
- swiley 11y ago
  You have obviously never used troff then.
- p4wnc6 11y ago
  That's not a good comparison, because mixing your own chemicals for photo development adds more time, effort, and defect rate to the process. Becoming more proficient with TeX saves time and makes for more re-usable typesetting structures, which allow you to accurately do what you want faster and faster. It's similar to using emacs or vim. You could say, "Why should programming devotees expect others to adopt those editors?" It's about productivity. I'm a productivity devotee and when I hear people use short-term thinking to discount how valuable the up-front costs for learning TeX (or emacs/vim, etc) are, it motivates me to help them see why it's not true. Of course, I don't want to be dogmatic either. If something works for you and your personal utility function discounts the value of things like TeX or emacs, that's fine.
  - amelius 11y ago
    > It's about productivity. No it is about aesthetics, which are mostly irrelevant for 99% of all research papers.
    - Spivak 11y ago
      It's not just aesthetics, having proper typesetting makes papers more accessible, easier to read and parse, the standard formatting makes understanding the layout of papers easier, there is a huge library of packages that cleanly abstract the task of rendering obscure symbols and complex mathematical formulas, and it automates the job of tracking figure, equation and reference citations to name just a few of the features. Fine, TeX is an ugly language that only a programmer could love, but in over twenty years nobody has yet been able to replace its expressive power and flexibility -- doc generators just compile to LaTeX and more user friendly GUIs like Lyx and TeXMaker have been written to ease the process of writing for end users, but the underlying format has yet to be replaced in over twenty years. Like Markdown, TeX rides the line between being able to be written by hand for greater flexibility and automatically generated for newbies. Unless you think that reading red text on a blue background in payprus would be an equally enjoyable reading experience it more than simple aesthetics.
      - deleted 11y ago
        [deleted]
      - wonderlust 11y ago
        ... did I just read a rant about how enjoyability and simplicity are NOT aesthetics?
      - ninguem2 11y ago
        Are you related to the author of "The Joy of Tex"?
      - smsm42 11y ago
        You sound like aesthetics is a bad thing you need to disavow for some reason. > Fine, TeX is an ugly language that only a programmer could love, but in over twenty years nobody has yet been able to replace its expressive power and flexibility Which doesn't make it less ugly. If we can't cure common cold yet, that is no reason to praise it and making it sound as if it's a great thing. It is a problem and so is the need to use ugly language to produce aesthetically pleasing papers. The fact that we don't have solution for this problem yet does not turn it into non-problem. It just says nobody yet was good enough to solve it.
- mchaver 11y ago
  At the very least it would make sense to provide the source that outputs the document. That could be something like OpenOffice or Latex. There are probably some WYSIWYG solution for Latex. If you are referencing a table from another paper and you want to show or alter it, it would be a lot quicker if you have the code that renders it. If you are in the business of writing papers that you want people to read and quote, it seems like a good idea to master the tools that help you do that.
  - dorfsmay 11y ago
    https://www.lyx.org/ https://www.lyx.org/
    - tormeh 11y ago
      LyX is salvation. Edit in WYSIWYG, export to latex or PDF.
- cbd1984 11y ago
  Good LaTeX is less about aesthetics and more about having boilerplate code and macros which greatly simplify the process of making readable documents with citations in the correct (nitpicky, archaic, utterly idiosyncratic) format and figures correctly included and labeled and captioned and scaled. The joy of LaTeX is that it handles the aesthetics for you. You can focus entirely on structure. Or that's how it works when you have good boilerplate to start from.
- gaur 11y ago
  I think the problem is that many people who don't use LaTeX (and some who do) are so inept at typography that it actually impedes the reader's ability to use the document. One LaTeX-agnostic example is when people abuse hyphens-like this-which makes it impossible to parse the sentence without reading it several times.
  - anigbrowl 11y ago
    Well you're onto something there. I liked typography for its own sake long before it became affordable to do it on personal computers, so I suppose I take my competence in that area for granted.
jessriedel 11y ago
FYI, the .tex source is available for all papers uploaded to the arXiv.
- semi-extrinsic 11y ago
  No, you can upload just the PDF if you want.
  - jessriedel 11y ago
    OK, correction: the .tex source is available for almost all papers uploaded to the arXiv (because they give a big warning/plea message to do so).
JohnHammersley 11y ago
You can now view (& compile) a large selection of latex examples, templates and articles directly in the browser on platforms such as Overleaf[1]. I'm one of the founders, and we're also making it possible to directly submit projects from Overleaf to repositories such as the arXiv and bioRxiv, and to traditional journals, to help speed up the submission and publication process.[2] [1] https://www.overleaf.com https://www.overleaf.com [2] https://www.overleaf.com/tutorial https://www.overleaf.com/tutorial

mentat 11y ago

Or, if published, we could develop tools to normalize the data and improve the state of the art.

chejazi 11y ago
This is the right approach. Like what OCR does for digitizing volumes of literature.
- dekhn 11y ago
  OCR for historical literature is a really hard problem. Beyond the fact that OCR quality is pretty low, perception of the meaning of the papers isn't addressed by that. So it makes it easier to access (can read papers on the net rather than going to a library) but that's about it.
  - chejazi 11y ago
    > it makes it easier to access (can read papers on the net rather than going to a library) but that's about it. I disagree. The text is indexed and searchable. > the meaning of the papers isn't addressed by that. I agree. I wonder if starting with research papers that have lots of symbolic logic (e.g. math proofs) would be the easiest starting point for a system like this.

jhbadger 11y ago

Also, (experimental) biologists tend not to use TeX/LaTeX. Typically they use Microsoft Word and a proprietary citation manager like Endnote. Typesetting is seen as something the journal deals with, not the author.

dekhn 11y ago
I was a biologist, but I used LaTeX: it was important to have parseable references in a flat file.
- leemailll 11y ago
  Many of biologists stick to word file, especially the review/communet function. It is often because the collaborators stick to word and you have to follow up with it.
  - dekhn 11y ago
    back in the day when I did this, I ended up writing papers with biologists. All the biologists would be cc'd on an email with the doc as an attachment and a "token" was passed between people to determine doc ownership. The person would make their changes and reattach the new doc to the email and pass the token to the next person. Ultimately, this would fall over as people accidentally made concurrent changes and it would be hard if not impossible to merge between two documents. Doc files were never designed to be put into a version control system with merge semantics. Some people use shared folders with locking but locking doesn't work well over file shares. I'm not sure but I suspect now people use dropbox instead of email; it's not clear to me how they manage conflicts and merges. I rather my format be a text-line-based non-rendered format that I can render down if need be, but use the full power of patch and diff to manage conflicts.

agumonkey 11y ago

Shame in 'science', I find that unbelievably absurd. About the 2nd part, may I rephrase it as PDFs are the docker of publication .. (Adobe would probably be very proud of that).

dekhn 11y ago

I wasn't defending LaTeX; I don't use it any more. I am certainly aware that many scientists don't want to publish their code out of embarrassment. It was a surprise to me, but I've heard that feedback. I investigated various alternatives after I found LaTeX too arcane, including DocBook. It's still not entirely clear what solution exists to the collection of problems I have (want to semantically mark up my paper, want it to display nicely regardless of screen, etc).

eru 11y ago
What about something like markdown plus some syntax for formulas? You can compile the markdown (via a detour to LaTeX perhaps) to something that looks nice.
- dekhn 11y ago
  Yeah, some form of enhanced markdown, with self-rendering papers on github (or whatever-hub or whatever-xiv) seems like a very practical solution. It couples the paper's representation to the version control system in a way that could be convenient and expressive.
  - cbd1984 11y ago
    I figure that any markdown dialect enhanced far enough to be usable would be as complex as LaTeX, only without the tooling and pre-written macros.
    - dorfsmay 11y ago
      Enhanced versions of markdown are just turning it back into a full wiki language.
  - dorfsmay 11y ago
    Markdown gets very limited very quickly. Are you aware of pandoc? http://pandoc.org/ http://pandoc.org/

beambot 11y ago

HA! Not to mention unflattering comments in LaTeX or BibTeX files. You thought source code had expletives... yeah.

vlasev 11y ago
Some of my .tex files may or may not contain a lot of profanities.

chriswarbo 11y ago

> Do you realize that many researchers would be ashamed if others saw their latex code? I really don't get this argument; I see it come up regarding open data for reproducibility too. TeX/LaTeX source is just an extra piece of metadata which might help someone out in a few years time. That's nothing to feel ashamed of, and it's certainly less shameful than not providing the code. If the code allows flaws in the research to be exposed, that's either something to be proud of, or (if it was intentional) it's not the code that's shameful. My own LaTeX is mostly an inconsistent mixture of copypasta from stackexchange. That doesn't mean I'm ashamed of it, and it's all in public Git repos at http://chriswarbo.net/git http://chriswarbo.net/git if any human or bot actually cares to look. Speaking of which, the commit histories of those Git repositories aren't very good either. Lots of commit messages like "Fixes", or "Oops, typo in last commit", with multiple changes bundled into single commits, atomic changes spread across multiple commits, etc. But, in the same way as my LaTeX source, it's better than having no commit history, which would be the case if I set myself higher standards. Of course, I can follow higher standards, but that's when I'm paid to and/or when collaboration is important. For personal stuff, I'd rather invest the time I save interacting with Git on writing more tests, or refactoring, etc.

slang800 11y ago

> Do you realize that many researchers would be ashamed if others saw their latex code? The exact same think could be said about HTML, but that hasn't stopped people...