3 ms·

I have been writing my thesis as an org-mode file in git that is automatically published [0] as a pdf every time I commit using a bash script tangled as a commi

by tgbugs 7y ago

I have been writing my thesis as an org-mode file in git that is automatically published [0] as a pdf every time I commit using a bash script tangled as a commit hook. Figures are (or can be) computed from live data, and org-ref can automatically manage and format all my references as I go without the insanity inducing workflows that one usually has to resort to. Another amazing thing about org-mode is that I can write a code block to hit a remote api (e.g. google docs) and fill and format entire chapters from a collaborative editing source that my advisor is comfortable using, C-c C-c and it is embedded in the document. Perfect for stitching papers together from a variety of sources. While there is a learning curve, the peace of mind of having a single tool that enables me to use any tool I need to use was such a relief after years of other painful writing and publishing workflows. If you are starting a PhD today and are going to do anything at all with code, I seriously suggest you learn org-mode. By the time you finish, not only will you have a PhD, but even if you don't you will have the power of one of the most amazing authoring tools under your command! (I would say, "Learn org-mode, simplify your life!" but emacs tends to lead to other complications). 0. https://orgmode.org/manual/Publishing.html https://orgmode.org/manual/Publishing.html

ska 7y ago
Back in the stone age when I did mine, my approach was more local, but similar. I wrote all the text in LaTeX using AucTeX package, but had a makefile and a commit hook (to CVS, later SVN, this was pre-git). All figures & tables etc. were generated from other make targets, which chained back to a mix of (usually) lisp code, matlab, or maple/mathematica. Oh, and figures generated by things like MetaPost. Structuring things at the chapter level .tex files made it all pretty manageable. I had figure dependencies on both input data and executable targets/scripts, so changing either would trigger a recompute, with the exception of some long running stuff - of course had global rebuild rules. I wrote many papers this way too, it was nice to know everything was synchronized properly. I did use org-mode to organize my work though. These days it would be on github and I probably wouldn't have lost it in a move.
kmundnic 7y ago
How do you deal with different sections from google docs? As in, how do you format these into markdown or latex sections? I've been working with psychologists and I've taught them the basics of a latex document, but I see how extracting the sections they have written from a google doc could be useful.
- tgbugs 7y ago
  I wrote a docs api v1 -> org converter in python. I just pulled it into its own repo and it still has a bunch of dependencies related to how I handle auth for other projects, but if you want to take a look it is at [0]. There is an example of how I use the Docs class in an org source bock in the readme (note that I use evil mode, and that you need to view raw to see everything that is going on with the headers). It currently requires a little dance to get everything in the right place at the right indent level, but it could be fully automated. I'm in the process of writing a converter going the other direction in elisp, and I will probably rewrite the python into elisp at some point as well, but for now, following the keystrokes is faster for me than trying to reimplement it to integrate with the org-mode api. As an overview I use the heading level in the google doc as the corresponding * level (e.g. h1 -> , h2 -> *, etc.). Nearly every construct in the google doc has a 1:1 mapping with some org mode convention (there are some things that I still write in org-syntax in the google doc, such as figure captions and stuff like that. If I need to export to markdown from org I usually try pandoc (I very rarely do this though). Importing from markdown I use pandoc as well, newer versions do a pretty good job and only need a few tweaks from time to time. Latex export is straight forward using org-mode's built in exporter [1] via the export dispatcher. For full pdf export I use lualatex via texlive with all the pain that that entails. 0. https://github.com/tgbugs/gdocorgpy https://github.com/tgbugs/gdocorgpy 1. https://orgmode.org/manual/LaTeX-export.html https://orgmode.org/manual/LaTeX-export.html 2. https://orgmode.org/manual/The-Export-Dispatcher.html#The-Export-Dispatcher https://orgmode.org/manual/The-Export-Dispatcher.html#The-Ex...
philsnow 7y ago
A while back when I still worked on academic papers, I had a great workflow where I had a Makefile that knew how to build everything (deriving .eps files from xfig diagrams, deriving a main pdf file from all the .tex files etc), and I used inotifytools to notice when any of the build dependencies changed, and that triggered a run of `make`. xpdf has (... had?? apparently it got removed in the 4.0 series to get a release out the door) a "remote" option where you can send commands to a running xpdf instance, so I would have the above makefile notify the running xpdf to re-read the file. Put $EDITOR on one side and xpdf on the other, and you've got a near-realtime-updated view of what you're working on, just save the file and see the rebuilt pdf. It usually took around a second or two depending on the complexity of what I was working on. I didn't use emacs at the time, but if I had, I would try to integrate all of the above with flymake-mode to automatically build without me needing to save the file. (there are probably problems with this because if it builds while you're in the middle of typing a latex command, the latex process will fail to build)

by tgbugs 7y ago

ska 7y ago

Back in the stone age when I did mine, my approach was more local, but similar. I wrote all the text in LaTeX using AucTeX package, but had a makefile and a commit hook (to CVS, later SVN, this was pre-git). All figures & tables etc. were generated from other make targets, which chained back to a mix of (usually) lisp code, matlab, or maple/mathematica. Oh, and figures generated by things like MetaPost. Structuring things at the chapter level .tex files made it all pretty manageable. I had figure dependencies on both input data and executable targets/scripts, so changing either would trigger a recompute, with the exception of some long running stuff - of course had global rebuild rules. I wrote many papers this way too, it was nice to know everything was synchronized properly. I did use org-mode to organize my work though. These days it would be on github and I probably wouldn't have lost it in a move.

kmundnic 7y ago

How do you deal with different sections from google docs? As in, how do you format these into markdown or latex sections? I've been working with psychologists and I've taught them the basics of a latex document, but I see how extracting the sections they have written from a google doc could be useful.

tgbugs 7y ago
I wrote a docs api v1 -> org converter in python. I just pulled it into its own repo and it still has a bunch of dependencies related to how I handle auth for other projects, but if you want to take a look it is at [0]. There is an example of how I use the Docs class in an org source bock in the readme (note that I use evil mode, and that you need to view raw to see everything that is going on with the headers). It currently requires a little dance to get everything in the right place at the right indent level, but it could be fully automated. I'm in the process of writing a converter going the other direction in elisp, and I will probably rewrite the python into elisp at some point as well, but for now, following the keystrokes is faster for me than trying to reimplement it to integrate with the org-mode api. As an overview I use the heading level in the google doc as the corresponding * level (e.g. h1 -> , h2 -> *, etc.). Nearly every construct in the google doc has a 1:1 mapping with some org mode convention (there are some things that I still write in org-syntax in the google doc, such as figure captions and stuff like that. If I need to export to markdown from org I usually try pandoc (I very rarely do this though). Importing from markdown I use pandoc as well, newer versions do a pretty good job and only need a few tweaks from time to time. Latex export is straight forward using org-mode's built in exporter [1] via the export dispatcher. For full pdf export I use lualatex via texlive with all the pain that that entails. 0. https://github.com/tgbugs/gdocorgpy https://github.com/tgbugs/gdocorgpy 1. https://orgmode.org/manual/LaTeX-export.html https://orgmode.org/manual/LaTeX-export.html 2. https://orgmode.org/manual/The-Export-Dispatcher.html#The-Export-Dispatcher https://orgmode.org/manual/The-Export-Dispatcher.html#The-Ex...

philsnow 7y ago

A while back when I still worked on academic papers, I had a great workflow where I had a Makefile that knew how to build everything (deriving .eps files from xfig diagrams, deriving a main pdf file from all the .tex files etc), and I used inotifytools to notice when any of the build dependencies changed, and that triggered a run of `make`. xpdf has (... had?? apparently it got removed in the 4.0 series to get a release out the door) a "remote" option where you can send commands to a running xpdf instance, so I would have the above makefile notify the running xpdf to re-read the file. Put $EDITOR on one side and xpdf on the other, and you've got a near-realtime-updated view of what you're working on, just save the file and see the rebuilt pdf. It usually took around a second or two depending on the complexity of what I was working on. I didn't use emacs at the time, but if I had, I would try to integrate all of the above with flymake-mode to automatically build without me needing to save the file. (there are probably problems with this because if it builds while you're in the middle of typing a latex command, the latex process will fail to build)