5 ms·

Announcing rkt v0.5, featuring pods, overlayfs, and more

74 points by TheHippo 12y ago

conradk 12y ago
I'm still wondering what the appc spec has more than Docker (and vice-versa). At the moment, I know Docker and find it easy to use. Why would I use Rocket instead ? Could anyone explain or link to non biased comparisons of the two, please ?
- vacri 12y ago
  Rocket isn't trying to have more than Docker, it's explicitly trying to have less. Rocket was started because the authors thought that Docker was losing sight of it's original mission and becoming bloatware. I've not used Rocket at all, but I'm having to implement Docker in production. As an ops guy rather than a dev, I'm finding a lot of rough edges. One example: I was away from the office a couple of days ago and one of the devs had to push to our staging servers instead of me. He logged in, and couldn't pull the image from dockerhub. Error message said the image was not found, which was demonstrably wrong, because he could pull it to his own machine. You just have to know that "image not found" can mean both "image not found" and "you haven't run 'docker login' yet" - that image not found error is given as an auth error message! There's stuff like this all through Docker, and I can see where the Rocket guys are coming from - Docker is spread too thin trying to do too much. Lots of corners get cut.
  - IanCal 12y ago
    > that image not found error is given as an auth error message! 404s are pretty common for that. Github, for example, returns "not found" pages for repos you can't access. Edit - however the login process is quite awkward, though set for improvement.
    - NeutronBoy 12y ago
      If only there were a separate error code to specifically indicate an authentication problem that could be returned...
      - ossreality 12y ago
        I'm all in favor of nitpicking some of Docker's awfulness, but returning a 400 for Bad Auth instead of 404 will leak information.
      - yrro 12y ago
        Github don't use the error code because to do so would leak information about the names of private repositories.
        derefr 12y ago
        It would be more correct, then, to signal an authentication error for all nonexistent-and-or-private repos. After all, you aren't authorized to know whether a repo exists by that name or not... whether or not one actually does. (This would also imply that organization owners (and Github CSRs and ops staff) would simply bypass that check, falling through to a check only for existence, where it would be appropriate to return 404. A similar reasoning is behind why you get a 403, not a 404, when you try to get the index of an empty S3 bucket. Sure, it doesn't exist—but you're also not allowed to know that.
        jacques_chester 12y ago
        Showing 404 to hide the existence of a resource is specifically called out as a suitable use in the RFCs: The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. http://tools.ietf.org/html/rfc7231#section-6.5.4 http://tools.ietf.org/html/rfc7231#section-6.5.4
        vacri 12y ago
        This being said, 403 is also valid in that same RFC, and makes more sense. "You are not authorised to know the status of this item" is more informative and less misleading than "This item doesn't exist". Everything should be auth-restricted by default (deny-by-default), except for items intentionally made public. In my example above, a 403 gives the correct nature of the fault without revealing any hidden information, whereas a 404 is demonstrably misleading.
- bryanlarsen 12y ago
  The enhancement I'm most looking forward to is the use of systemd for process management. Docker handles that itself, and can lose track if the process forks strangely and/or badly.
  - shazow 12y ago
    Like postfix. :/ People tend to wrap it with supervisord which does a better job at tracking child processes.
  - deleted 12y ago
    [deleted]
josh2600 12y ago
As a very effective substrate for hosting containers, we at Terminal think a lot about which of the container mechanisms will win over the long term. My personal opinion is that the winner will be the group that successfully gets Enterprises to change their workload design. I also don't think that Rocket is necessarily a superior format to Docker, but I think they're both dealing with the recognition that any big change in Enterprise behavior represents an opportunity for value capture. There's a real question as to where any of these abstraction layers fit in if Docker wins (and there's some possibility docker is going to win). If that's the case, coreos doesn't want to look back a few years down the road and wish they'd been working on a container format. It's 2015. One of the battlegrounds for enterprise dollars is containers. It's going to be a delightful thing to watch. It's also worth noting that many companies have their own cgroups implementations which are neither docker or rocket based. I rather like the position of dispassionate observer in this war (at Terminal we run all of the containers and also apps without containers).
- jacques_chester 12y ago
  > It's also worth noting that many companies have their own cgroups implementations which are neither docker or rocket based. See for example Garden (née Warden)[0][1], which is the basic building block of Cloud Foundry. [0] https://github.com/cloudfoundry-incubator/garden https://github.com/cloudfoundry-incubator/garden [1] http://blog.pivotal.io/cloud-foundry-pivotal/features/cloud-foundry-container-technology-a-garden-overview http://blog.pivotal.io/cloud-foundry-pivotal/features/cloud-...
krakensden 12y ago
It's weird seeing all the careful description of a Pod, which is just an explicit rollback of Docker's wishful thinking about single-process containers to something identical to what you'd get with LXC. Which just runs /sbin/init. The amount of wasted development effort caused by Docker's willful intransigence on this is sort of staggering. The teams I work with using docker are still tripping over new nonsense that would have been solved by... I don't know, using systemd... after almost a year of trying to work through the kinks.
- wmf 12y ago
  I can imagine that composing a pod out of container images could have advantages over composing a system out of packages because communication between the containers would be more explicit.
  - krakensden 12y ago
    I see it as a hassle- I dont want to be doing a bunch or redundant whitelisting to make the percona tools work. Worse, I don't want to ship them separately, xtrabackup uses mysql as a library, which means separate containers only buys you bugs. Docker's design decisions make sense if you're shipping statically linked, standalone binaries everywhere. Which I suspect docker.io and many other people are doing. But that's also sort of a boring edge case where you don't even need filesystem namespaces except for cleanliness.
zalmoxes 12y ago
For reference: the kubernetes guys have an open proposal for pods in Docker since October: https://github.com/docker/docker/issues/8781 https://github.com/docker/docker/issues/8781
- derefr 12y ago
  And even without them, if you develop using fig you're pretty much writing pod manifests.
zobzu 12y ago
This is cool. Overlayfs IS a huge difference in performance due to fs level caching amount namespaces. Hopefully btrfs will get that too, some day. I must say however, I dont like the pod concept too much. That locks you in a bit. Additionally and in a different register, the deployment systems, while these work fine when you have to the time to do things right, dont work so well in practice. All companies that I know of which have 1k employes or less (ie most) basically dont update anything automatically because the redeployment might still fail. And of course, manual labor costs a lot of human resources. We still need a better way to separate the system update process from the deployment, settings, and app.
- krakensden 12y ago
  > I must say however, I dont like the pod concept too much. That locks you in a bit. They didn't make it up, it's straight from Kubernetes. Presumably the Docker team will wind up incorporating something similar soonish, as well. That said, it's basically a fancy pants way of saying "we are going to run multiple processes in the same namespace", and if you're using runit like so many people are, you're already doing it.
- jzxcv 12y ago
  Could you explain what you mean by "locks you in"?
  - jsprogrammer 12y ago
    You now have to run all those programs in the same namespace, on the same machine. There is no possibility to run some of the programs somewhere else (without building a new image).
    - tb93 12y ago
      No, pod is a collection of multiple images, not a single fat one.
      - jsprogrammer 12y ago

74 points by TheHippo 12y ago

I'm still wondering what the appc spec has more than Docker (and vice-versa). At the moment, I know Docker and find it easy to use. Why would I use Rocket instead ? Could anyone explain or link to non biased comparisons of the two, please ?

vacri 12y ago
Rocket isn't trying to have more than Docker, it's explicitly trying to have less. Rocket was started because the authors thought that Docker was losing sight of it's original mission and becoming bloatware. I've not used Rocket at all, but I'm having to implement Docker in production. As an ops guy rather than a dev, I'm finding a lot of rough edges. One example: I was away from the office a couple of days ago and one of the devs had to push to our staging servers instead of me. He logged in, and couldn't pull the image from dockerhub. Error message said the image was not found, which was demonstrably wrong, because he could pull it to his own machine. You just have to know that "image not found" can mean both "image not found" and "you haven't run 'docker login' yet" - that image not found error is given as an auth error message! There's stuff like this all through Docker, and I can see where the Rocket guys are coming from - Docker is spread too thin trying to do too much. Lots of corners get cut.
- IanCal 12y ago
  > that image not found error is given as an auth error message! 404s are pretty common for that. Github, for example, returns "not found" pages for repos you can't access. Edit - however the login process is quite awkward, though set for improvement.
  - NeutronBoy 12y ago
    If only there were a separate error code to specifically indicate an authentication problem that could be returned...
    - ossreality 12y ago
      I'm all in favor of nitpicking some of Docker's awfulness, but returning a 400 for Bad Auth instead of 404 will leak information.
    - yrro 12y ago
      Github don't use the error code because to do so would leak information about the names of private repositories.
      - derefr 12y ago
        It would be more correct, then, to signal an authentication error for all nonexistent-and-or-private repos. After all, you aren't authorized to know whether a repo exists by that name or not... whether or not one actually does. (This would also imply that organization owners (and Github CSRs and ops staff) would simply bypass that check, falling through to a check only for existence, where it would be appropriate to return 404. A similar reasoning is behind why you get a 403, not a 404, when you try to get the index of an empty S3 bucket. Sure, it doesn't exist—but you're also not allowed to know that.
        jacques_chester 12y ago
        Showing 404 to hide the existence of a resource is specifically called out as a suitable use in the RFCs: The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. http://tools.ietf.org/html/rfc7231#section-6.5.4 http://tools.ietf.org/html/rfc7231#section-6.5.4
        vacri 12y ago
        This being said, 403 is also valid in that same RFC, and makes more sense. "You are not authorised to know the status of this item" is more informative and less misleading than "This item doesn't exist". Everything should be auth-restricted by default (deny-by-default), except for items intentionally made public. In my example above, a 403 gives the correct nature of the fault without revealing any hidden information, whereas a 404 is demonstrably misleading.
bryanlarsen 12y ago
The enhancement I'm most looking forward to is the use of systemd for process management. Docker handles that itself, and can lose track if the process forks strangely and/or badly.
- shazow 12y ago
  Like postfix. :/ People tend to wrap it with supervisord which does a better job at tracking child processes.
- deleted 12y ago
  [deleted]

josh2600 12y ago

As a very effective substrate for hosting containers, we at Terminal think a lot about which of the container mechanisms will win over the long term. My personal opinion is that the winner will be the group that successfully gets Enterprises to change their workload design. I also don't think that Rocket is necessarily a superior format to Docker, but I think they're both dealing with the recognition that any big change in Enterprise behavior represents an opportunity for value capture. There's a real question as to where any of these abstraction layers fit in if Docker wins (and there's some possibility docker is going to win). If that's the case, coreos doesn't want to look back a few years down the road and wish they'd been working on a container format. It's 2015. One of the battlegrounds for enterprise dollars is containers. It's going to be a delightful thing to watch. It's also worth noting that many companies have their own cgroups implementations which are neither docker or rocket based. I rather like the position of dispassionate observer in this war (at Terminal we run all of the containers and also apps without containers).

jacques_chester 12y ago
> It's also worth noting that many companies have their own cgroups implementations which are neither docker or rocket based. See for example Garden (née Warden)[0][1], which is the basic building block of Cloud Foundry. [0] https://github.com/cloudfoundry-incubator/garden https://github.com/cloudfoundry-incubator/garden [1] http://blog.pivotal.io/cloud-foundry-pivotal/features/cloud-foundry-container-technology-a-garden-overview http://blog.pivotal.io/cloud-foundry-pivotal/features/cloud-...

krakensden 12y ago

It's weird seeing all the careful description of a Pod, which is just an explicit rollback of Docker's wishful thinking about single-process containers to something identical to what you'd get with LXC. Which just runs /sbin/init. The amount of wasted development effort caused by Docker's willful intransigence on this is sort of staggering. The teams I work with using docker are still tripping over new nonsense that would have been solved by... I don't know, using systemd... after almost a year of trying to work through the kinks.

wmf 12y ago
I can imagine that composing a pod out of container images could have advantages over composing a system out of packages because communication between the containers would be more explicit.
- krakensden 12y ago
  I see it as a hassle- I dont want to be doing a bunch or redundant whitelisting to make the percona tools work. Worse, I don't want to ship them separately, xtrabackup uses mysql as a library, which means separate containers only buys you bugs. Docker's design decisions make sense if you're shipping statically linked, standalone binaries everywhere. Which I suspect docker.io and many other people are doing. But that's also sort of a boring edge case where you don't even need filesystem namespaces except for cleanliness.

zalmoxes 12y ago

For reference: the kubernetes guys have an open proposal for pods in Docker since October: https://github.com/docker/docker/issues/8781 https://github.com/docker/docker/issues/8781

derefr 12y ago
And even without them, if you develop using fig you're pretty much writing pod manifests.

zobzu 12y ago

This is cool. Overlayfs IS a huge difference in performance due to fs level caching amount namespaces. Hopefully btrfs will get that too, some day. I must say however, I dont like the pod concept too much. That locks you in a bit. Additionally and in a different register, the deployment systems, while these work fine when you have to the time to do things right, dont work so well in practice. All companies that I know of which have 1k employes or less (ie most) basically dont update anything automatically because the redeployment might still fail. And of course, manual labor costs a lot of human resources. We still need a better way to separate the system update process from the deployment, settings, and app.

krakensden 12y ago
> I must say however, I dont like the pod concept too much. That locks you in a bit. They didn't make it up, it's straight from Kubernetes. Presumably the Docker team will wind up incorporating something similar soonish, as well. That said, it's basically a fancy pants way of saying "we are going to run multiple processes in the same namespace", and if you're using runit like so many people are, you're already doing it.
jzxcv 12y ago
Could you explain what you mean by "locks you in"?
- jsprogrammer 12y ago
  You now have to run all those programs in the same namespace, on the same machine. There is no possibility to run some of the programs somewhere else (without building a new image).
  - tb93 12y ago
    No, pod is a collection of multiple images, not a single fat one.
    - jsprogrammer 12y ago