7 ms·

I had just landed LXC container support in Velociraptor [1] when Docker was announced last year. It uses Supervisor to launch LXC containers and run your app i

by bjt 12y ago

I had just landed LXC container support in Velociraptor [1] when Docker was announced last year. It uses Supervisor to launch LXC containers and run your app inside. I thought long and hard about switching to Docker, but their decision to remove standalone mode [2] would have meant replacing all of Velociraptor's Supervisor integration with Docker integration instead. With Docker being such a moving target over that time span, it just seemed like a bad move. Since then I've been mulling writing my own standalone 'drydock' utility that would just start a single container and then get out of the way (as opposed to the Docker daemon that insists on being the parent of everything). I'm optimistic that Rocket could be that thing. Question though: Does Rocket have any concept of the image layering that Docker does? That still seems to me like a killer feature. [1] https://bitbucket.org/yougov/velociraptor/ https://bitbucket.org/yougov/velociraptor/ [2] https://github.com/docker/docker/issues/503 https://github.com/docker/docker/issues/503

philips 12y ago
Yes, the app-container spec has the concept of dependent filesets. See: https://github.com/coreos/rocket/blob/master/app-container/SPEC.md#fileset-images https://github.com/coreos/rocket/blob/master/app-container/S... https://github.com/coreos/rocket/blob/master/app-container/SPEC.md#fileset-manifest https://github.com/coreos/rocket/blob/master/app-container/S... What do you think of the filesets concept?
- bjt 12y ago
  I'm still digesting the Go-like syntax for vanity URLs and how that works here. If a fileset manifest lets you specify the URLs where the layers can be fetched from, then I like it. Does Rocket just 'cp' files on top of each other to implement layering? It'd be nice to not require a bunch of copies of the same files. I thought that the hard link implementation in Docker's new overlayfs support was a smart idea.
  - philips 12y ago
    Yes, all of this was designed with overlayfs in mind. I am waiting anxiously for Linux Kernel 3.18 to land, this is a huge step forward for Linux and years in the making.
    - bjt 12y ago
      Whether Velociraptor uses Rocket or not, implementing the App Container Spec seems like a no brainer. I've made https://bitbucket.org/yougov/velociraptor/issue/136 https://bitbucket.org/yougov/velociraptor/issue/136.
    - sophacles 12y ago
      Cool! The idea of filesets is very nice - there are some very interesting workflow ideas buried in there. I've been looking forward to a mainstream unioning filesystem for a while too - and I hope rocket does some serious exploration (or enables it) of how to take advantage of them fully in both development and deployment. (And while I'm at it, testing too). My personal wish-list down this path includes: * options in the relevant manifests on which layer is writable ... if I'm doing development on libfoo which is used by several different apps, let me make that layer writable so I can rapidly iterate integration tests and (bad practice) live coding on testing/dev servers. * tools to help me smash a dev layer or 3 into a single fileset (and similarly dissect a layer into a few new filesets during a refactoring) * the ability to use filesets and overlays in a way similar to package management is now, but with extended features that are similar to python's virtualenv. One of the things I see as a boon of the filesets as described is: I can update parts of my system without having to rebuild the whole dang app silo from the get go. Combining this with some of the above features looks like it could be useful for making "thin" images - where I can build all my code in one place, and port only the binaries to the staging and deployment images, just by doing a few fileset/overlay tricks. (no more complicated scripts)
- digitalzombie 12y ago
  Please no json file, use yaml or have an option for yaml T___T. The curly braces and brackets can get ugly when nested. edit: It may seems stupid but when you're on the terminal with vim and the directory path is long in some nested object/array. Things is really hard to parse out with your eyes.
  - ecnahc515 12y ago
    I think using JSON is a solid choice. You can easily make yaml to JSON translators for this purpose.
    - ash 12y ago
      JSON is okay, but I hope TOML will gain traction soon. Because comments. And trailing commas. Rust Crate is already using TOML for package metadata. https://github.com/toml-lang/toml https://github.com/toml-lang/toml http://doc.crates.io/manifest.html http://doc.crates.io/manifest.html
      - bjt 12y ago
        I've been -1 on TOML for a long time, and preferred YAML, but your point about comments is persuading me.
      - nor0101 12y ago
        +1 for TOML, started playing with Rust a couple of weeks ago.
      - catwell 12y ago
        +1 for TOML. If you think simplicity matters, look at the YAML spec [1], vomit and never use it again. If you still don't believe me, try implementing a conforming YAML 1.2 parser (oh, and try not to make it a source of RCE [2]...). [1] http://yaml.org/spec/1.2/spec.html http://yaml.org/spec/1.2/spec.html [2] http://blog.codeclimate.com/blog/2013/01/10/rails-remote-code-execution-vulnerability-explained/ http://blog.codeclimate.com/blog/2013/01/10/rails-remote-cod...
      - 12y ago
sciurus 12y ago
For running Docker under Supervisor, you may get some inspiration from https://github.com/ibuildthecloud/systemd-docker https://github.com/ibuildthecloud/systemd-docker

by bjt 12y ago

philips 12y ago

Yes, the app-container spec has the concept of dependent filesets. See: https://github.com/coreos/rocket/blob/master/app-container/SPEC.md#fileset-images https://github.com/coreos/rocket/blob/master/app-container/S... https://github.com/coreos/rocket/blob/master/app-container/SPEC.md#fileset-manifest https://github.com/coreos/rocket/blob/master/app-container/S... What do you think of the filesets concept?

bjt 12y ago
I'm still digesting the Go-like syntax for vanity URLs and how that works here. If a fileset manifest lets you specify the URLs where the layers can be fetched from, then I like it. Does Rocket just 'cp' files on top of each other to implement layering? It'd be nice to not require a bunch of copies of the same files. I thought that the hard link implementation in Docker's new overlayfs support was a smart idea.
- philips 12y ago
  Yes, all of this was designed with overlayfs in mind. I am waiting anxiously for Linux Kernel 3.18 to land, this is a huge step forward for Linux and years in the making.
  - bjt 12y ago
    Whether Velociraptor uses Rocket or not, implementing the App Container Spec seems like a no brainer. I've made https://bitbucket.org/yougov/velociraptor/issue/136 https://bitbucket.org/yougov/velociraptor/issue/136.
  - sophacles 12y ago
    Cool! The idea of filesets is very nice - there are some very interesting workflow ideas buried in there. I've been looking forward to a mainstream unioning filesystem for a while too - and I hope rocket does some serious exploration (or enables it) of how to take advantage of them fully in both development and deployment. (And while I'm at it, testing too). My personal wish-list down this path includes: * options in the relevant manifests on which layer is writable ... if I'm doing development on libfoo which is used by several different apps, let me make that layer writable so I can rapidly iterate integration tests and (bad practice) live coding on testing/dev servers. * tools to help me smash a dev layer or 3 into a single fileset (and similarly dissect a layer into a few new filesets during a refactoring) * the ability to use filesets and overlays in a way similar to package management is now, but with extended features that are similar to python's virtualenv. One of the things I see as a boon of the filesets as described is: I can update parts of my system without having to rebuild the whole dang app silo from the get go. Combining this with some of the above features looks like it could be useful for making "thin" images - where I can build all my code in one place, and port only the binaries to the staging and deployment images, just by doing a few fileset/overlay tricks. (no more complicated scripts)
digitalzombie 12y ago
Please no json file, use yaml or have an option for yaml T___T. The curly braces and brackets can get ugly when nested. edit: It may seems stupid but when you're on the terminal with vim and the directory path is long in some nested object/array. Things is really hard to parse out with your eyes.
- ecnahc515 12y ago
  I think using JSON is a solid choice. You can easily make yaml to JSON translators for this purpose.
  - ash 12y ago
    JSON is okay, but I hope TOML will gain traction soon. Because comments. And trailing commas. Rust Crate is already using TOML for package metadata. https://github.com/toml-lang/toml https://github.com/toml-lang/toml http://doc.crates.io/manifest.html http://doc.crates.io/manifest.html
    - bjt 12y ago
      I've been -1 on TOML for a long time, and preferred YAML, but your point about comments is persuading me.
    - nor0101 12y ago
      +1 for TOML, started playing with Rust a couple of weeks ago.
    - catwell 12y ago
      +1 for TOML. If you think simplicity matters, look at the YAML spec [1], vomit and never use it again. If you still don't believe me, try implementing a conforming YAML 1.2 parser (oh, and try not to make it a source of RCE [2]...). [1] http://yaml.org/spec/1.2/spec.html http://yaml.org/spec/1.2/spec.html [2] http://blog.codeclimate.com/blog/2013/01/10/rails-remote-code-execution-vulnerability-explained/ http://blog.codeclimate.com/blog/2013/01/10/rails-remote-cod...
    - 12y ago

sciurus 12y ago

For running Docker under Supervisor, you may get some inspiration from https://github.com/ibuildthecloud/systemd-docker https://github.com/ibuildthecloud/systemd-docker