5 ms·

Intro to Docker and how we use it at WriteLaTeX

157 points by jdleesmiller 12y ago

plg 12y ago
q: what did you use to generate the slides? edit: I see: cleaver
- jdleesmiller 12y ago
  They're generated by cleaver: https://github.com/jdan/cleaver https://github.com/jdan/cleaver You can see how it's set up in the github repo for the slides: https://github.com/jdleesmiller/ds-docker-demo https://github.com/jdleesmiller/ds-docker-demo
- jonalmeida 12y ago
  Another option is Brian Cavalier's slide deck format which is similar and easy to use. https://github.com/briancavalier/slides https://github.com/briancavalier/slides
errordeveloper 12y ago
> Each compile job gets its own short-lived container Just curious, how do handle input and output? Do you prepare a volume with the input and then grab output from there as well, or the program inside the container is fetching input from somewhere first, runs exec latex and then uploads the output elsewhere?
- jdleesmiller 12y ago
  Yes, we use a volume to get files in and out of the container, and then we upload the relevant files to Cloud Files for the client.
xtrumanx 12y ago
Yay, the first Docker post I can understand. I just installed and used Docker today to get a simple web server running out of a VPS (which is also a first for me as I always used managed services like Heroku). However, some aspects of Docker leave me with concerns that it may not be the tool for me. I tried installing the official node image and it downloaded hundreds of megabytes of other images (probably over a GB). Not that much of a problem on my VPS but absolutely unusable anywhere else in my corner of the world. Looks like I'll have to create my own images and use my own private local registry to make use of Docker outside of my VPS.
- jdleesmiller 12y ago
  I'm very glad to hear that it was understandable! I can't say I've used many of the community images. We generally just use the base OS images and install packages as needed. The Dockerfile makes it pretty easy to do.
- incision 12y ago
  Yeah, those are layers in the image. That image [1] (727MB) is built on buildpack-deps [2] (695MB) which is itself built on the base jessie (154MB). It would be pretty straightforward to adapt those Dockerfiles to create an image which includes only the dependencies for building node. It would end up looking a lot like 'node:slim' [3] (288MB). Ideally, Docker will eventually have the functionality to more easily strip out transient requirements like build dependencies from the final images. 1: https://github.com/docker-library/node/blob/013858ac35afb9ca7b10102956427b629e7708da/0.10/Dockerfile https://github.com/docker-library/node/blob/013858ac35afb9ca... 2: https://github.com/docker-library/buildpack-deps/blob/a201b127ddf29f47738a9fd04b2f004eb41b009f/jessie/Dockerfile https://github.com/docker-library/buildpack-deps/blob/a201b1... 3: https://github.com/docker-library/node/blob/013858ac35afb9ca7b10102956427b629e7708da/0.10/slim/Dockerfile https://github.com/docker-library/node/blob/013858ac35afb9ca...
qsymmachus 12y ago
Really nice presentation. Thanks for sharing this!
JamesMck 12y ago
clear and concise. thx for sharing!
rcchen 12y ago
Great presentation, it was very easy to follow and understand. Quick question though: is the only reason for using Docker containers with LaTeX file compilation for providing isolation between documents? Isn't there a performance hit versus running file compilation directly on worker machines, perhaps with some sort of folder-based isolation (workers will only compile files in folders that the user has permissions to)?
- jdleesmiller 12y ago
  Thanks! Isolation is definitely one of the main benefits for us. Compared to e.g. a chroot, docker also lets us disable networking and restrict memory etc. for the process in the container. It's another important layer of security. Another benefit is that the Dockerfile also makes it a lot easier to manage installation of all the LaTeX packages, fonts and various scientific software that we have installed. The overheads seem to be very low --- less than 100ms extra startup and tear down time, and no significant difference in runtime speed.
frequentflyeru 12y ago
Love the format of your presentation. I wish all web based presentations were like that.
j_s 12y ago
Docker as a layer of security vs. untrusted user input being compiled as root? I'd be interested in hearing more about the decision-making since Docker doesn't claim to provide protection from untrusted users.
- jdleesmiller 12y ago
  Running untrusted programs as root in a docker container is definitely not recommended (and indeed you should be very wary of running anything as root). In the slides, I didn't have time to go into how to run processes in containers without root privileges, but docker does provide features for doing so, and it's definitely worth using them.
  - bmh100 12y ago
    Would you please consider modifying or extending this presentation to include other basics? You did a great job at making Docker finally understandable as a whole. I'd love to see more of your writing on this topic.
    - jdleesmiller 12y ago
      Thanks! I will see what I can do to add more. Note that the source for the slides is on github [1], so anyone who'd like to pitch in can do so, and that would be most welcome. [1] https://github.com/jdleesmiller/ds-docker-demo https://github.com/jdleesmiller/ds-docker-demo
adiM 12y ago
Would it be possible for you to add support to other TeX formats, in particular, ConTeXt?
- jdleesmiller 12y ago
  Yes, we are planning to support ConTeXt on writeLaTeX. I think you can vote for it (or add it) on our feedback forum: writelatex.uservoice.com.
hackerboos 12y ago
I disagree with running RVM in a docker container. The point of RVM is to manage multiple versions of ruby and a container is only meant to have a single responsibility so running multiple versions of ruby is not advised. I think you're better off installing the required version with Ruby Install [1] rather than adding the complexity of a version manager. [1] - https://github.com/postmodern/ruby-install#readme https://github.com/postmodern/ruby-install#readme
- jdleesmiller 12y ago
  That's a fair point. I guess I recommended RVM by default, since it's what I usually use. Pull requests welcome! [1] [1] https://github.com/jdleesmiller/ds-docker-demo https://github.com/jdleesmiller/ds-docker-demo

157 points by jdleesmiller 12y ago

plg 12y ago

q: what did you use to generate the slides? edit: I see: cleaver

jdleesmiller 12y ago
They're generated by cleaver: https://github.com/jdan/cleaver https://github.com/jdan/cleaver You can see how it's set up in the github repo for the slides: https://github.com/jdleesmiller/ds-docker-demo https://github.com/jdleesmiller/ds-docker-demo
jonalmeida 12y ago
Another option is Brian Cavalier's slide deck format which is similar and easy to use. https://github.com/briancavalier/slides https://github.com/briancavalier/slides

errordeveloper 12y ago

> Each compile job gets its own short-lived container Just curious, how do handle input and output? Do you prepare a volume with the input and then grab output from there as well, or the program inside the container is fetching input from somewhere first, runs exec latex and then uploads the output elsewhere?

jdleesmiller 12y ago
Yes, we use a volume to get files in and out of the container, and then we upload the relevant files to Cloud Files for the client.

xtrumanx 12y ago

Yay, the first Docker post I can understand. I just installed and used Docker today to get a simple web server running out of a VPS (which is also a first for me as I always used managed services like Heroku). However, some aspects of Docker leave me with concerns that it may not be the tool for me. I tried installing the official node image and it downloaded hundreds of megabytes of other images (probably over a GB). Not that much of a problem on my VPS but absolutely unusable anywhere else in my corner of the world. Looks like I'll have to create my own images and use my own private local registry to make use of Docker outside of my VPS.

jdleesmiller 12y ago
I'm very glad to hear that it was understandable! I can't say I've used many of the community images. We generally just use the base OS images and install packages as needed. The Dockerfile makes it pretty easy to do.
incision 12y ago
Yeah, those are layers in the image. That image [1] (727MB) is built on buildpack-deps [2] (695MB) which is itself built on the base jessie (154MB). It would be pretty straightforward to adapt those Dockerfiles to create an image which includes only the dependencies for building node. It would end up looking a lot like 'node:slim' [3] (288MB). Ideally, Docker will eventually have the functionality to more easily strip out transient requirements like build dependencies from the final images. 1: https://github.com/docker-library/node/blob/013858ac35afb9ca7b10102956427b629e7708da/0.10/Dockerfile https://github.com/docker-library/node/blob/013858ac35afb9ca... 2: https://github.com/docker-library/buildpack-deps/blob/a201b127ddf29f47738a9fd04b2f004eb41b009f/jessie/Dockerfile https://github.com/docker-library/buildpack-deps/blob/a201b1... 3: https://github.com/docker-library/node/blob/013858ac35afb9ca7b10102956427b629e7708da/0.10/slim/Dockerfile https://github.com/docker-library/node/blob/013858ac35afb9ca...

qsymmachus 12y ago

Really nice presentation. Thanks for sharing this!

JamesMck 12y ago

clear and concise. thx for sharing!

rcchen 12y ago

Great presentation, it was very easy to follow and understand. Quick question though: is the only reason for using Docker containers with LaTeX file compilation for providing isolation between documents? Isn't there a performance hit versus running file compilation directly on worker machines, perhaps with some sort of folder-based isolation (workers will only compile files in folders that the user has permissions to)?

jdleesmiller 12y ago
Thanks! Isolation is definitely one of the main benefits for us. Compared to e.g. a chroot, docker also lets us disable networking and restrict memory etc. for the process in the container. It's another important layer of security. Another benefit is that the Dockerfile also makes it a lot easier to manage installation of all the LaTeX packages, fonts and various scientific software that we have installed. The overheads seem to be very low --- less than 100ms extra startup and tear down time, and no significant difference in runtime speed.

frequentflyeru 12y ago

Love the format of your presentation. I wish all web based presentations were like that.

j_s 12y ago

Docker as a layer of security vs. untrusted user input being compiled as root? I'd be interested in hearing more about the decision-making since Docker doesn't claim to provide protection from untrusted users.

jdleesmiller 12y ago
Running untrusted programs as root in a docker container is definitely not recommended (and indeed you should be very wary of running anything as root). In the slides, I didn't have time to go into how to run processes in containers without root privileges, but docker does provide features for doing so, and it's definitely worth using them.
- bmh100 12y ago
  Would you please consider modifying or extending this presentation to include other basics? You did a great job at making Docker finally understandable as a whole. I'd love to see more of your writing on this topic.
  - jdleesmiller 12y ago
    Thanks! I will see what I can do to add more. Note that the source for the slides is on github [1], so anyone who'd like to pitch in can do so, and that would be most welcome. [1] https://github.com/jdleesmiller/ds-docker-demo https://github.com/jdleesmiller/ds-docker-demo

adiM 12y ago

Would it be possible for you to add support to other TeX formats, in particular, ConTeXt?

jdleesmiller 12y ago
Yes, we are planning to support ConTeXt on writeLaTeX. I think you can vote for it (or add it) on our feedback forum: writelatex.uservoice.com.

hackerboos 12y ago

I disagree with running RVM in a docker container. The point of RVM is to manage multiple versions of ruby and a container is only meant to have a single responsibility so running multiple versions of ruby is not advised. I think you're better off installing the required version with Ruby Install [1] rather than adding the complexity of a version manager. [1] - https://github.com/postmodern/ruby-install#readme https://github.com/postmodern/ruby-install#readme

jdleesmiller 12y ago
That's a fair point. I guess I recommended RVM by default, since it's what I usually use. Pull requests welcome! [1] [1] https://github.com/jdleesmiller/ds-docker-demo https://github.com/jdleesmiller/ds-docker-demo