22 ms·

Open Container Initiative specifications are 1.0

217 points by philips 9y ago

virgil_disgr4ce 9y ago
What happens if I get pulled over with this spec in my car? X-)
- sctb 9y ago
  We've asked you already to please stop posting unsubstantively like this, and we ban accounts that refuse.
- virgil_disgr4ce 9y ago
  No jokes allowed I guess :)
chubot 9y ago
Does anyone have experience with non-Docker OCI implementations to share? What's an example of something that uses OCI but not Docker? The blog post is a little vague. I applaud this effort because Docker is sloppy, but I also realize that this isn't usually how technological adoption occurs. Usually the sloppy implementation is standardized, e.g. JavaScript, POSIX, CSV, Markdown, Ruby and Python to some extent -- rather than something designed from scratch or designed with fewer implementation artifacts. Maybe C++ is an exception, where things are actually standardized and there are multiple, healthy, competing implementations.
- philips 9y ago
  See the implementation lists for both specs: https://github.com/opencontainers/image-spec/blob/master/implementations.md https://github.com/opencontainers/image-spec/blob/master/imp... https://github.com/opencontainers/runtime-spec/blob/master/implementations.md https://github.com/opencontainers/runtime-spec/blob/master/i... Now that things are 1.0 we expect many people to start implementing the spec. For example, the transition steps required to add OCI v1.0 support on-top of Docker v2.2 support for an application primarily involves changing a few strings. See the implementation notes here. https://github.com/opencontainers/image-spec/blob/6060c3b6c6c81bdda7b113bc17299bab6633aa1c/media-types.md https://github.com/opencontainers/image-spec/blob/6060c3b6c6...
- pjmlp 9y ago
  > Maybe C++ is an exception, where things are actually standardized and there are multiple, healthy, competing implementations. I see you missed the 1990 and early 2000's part of C++ history.
  - chubot 9y ago
    Why don't gcc and MS C++ count? I think gcc was a little behind yes, but it caught up. Also I think there were several other independent C++ vendors in the 90's, whose names I don't recall.
    - pjmlp 9y ago
      Just out of my mind, as an early C++ adopter (1993). Turbo C++, Borland C++, Watcom C++, Green Hills C++, Microsoft C++ 7.0, djgpp for MS-DOS. Turbo C++, Borland C++, Watcom C++, Green Hills C++, Microsoft C++ 7.0, C++ Builder, Symantec C++, Metrowerks C++, C++ Builder, Visual C++ for Windows 3.x onwards. MPW C++, Metrowerks C++, Visual C++ for Mac OS. Borland C++, CSet++, Visual Age for C++ for OS/2. aC++ for HP-UX, xlC++ for Aix, Sun Forte C++ for Solaris. gcc for most UNIX systems. These are just the ones I remember without having to do a search. Writing portable C++ code in those days was quite an adventure, specially since we only had the CFront followed by C++ ARM as standards, while ANSI/ISO were working on the first actual standard.
      - chubot 9y ago
        Sure, but you're not really disagreeing with me. I was saying that C++ wasn't dominated by a particular implementation, i.e. it wasn't a "fake standard". It was and is a standard that MANY independent implementations tried to conform to!
- bkeroack 9y ago
  > Usually the sloppy implementation is standardized That's what this is. OCI v1.0 is pretty much "what Docker does".
  - cyphar 9y ago
    While this is somewhat true for the image-spec (with some very notable exceptions where the semantics have deviated heavily), it's absolutely untrue for the runtime-spec. As one of the people who maintains the legacy of what Docker did before the OCI (libcontainer which is now part of runc), the interfaces and semantics are very significantly different. Docker does use the OCI runtime-spec internally now (with plans to support the image-spec at some point), but that happened after the spec had become fairly stable. The most important thing is not where the spec design came from, but that it's now being maintained by a community that will make sure you don't end up with fragmentation and lock-in.
    - pcthrowaway 9y ago
      How much does the OCI image spec differ from the one developed by CoreOS for appc? When OCI was announced, image spec was outside of its scope (https://coreos.com/blog/making-sense-of-standards.html https://coreos.com/blog/making-sense-of-standards.html), so coreos's work in this space seemed highly important. So has all work essentially been abandoned? Or did it inform the image spec of OCI (and 'what docker does') at all?
  - scaryclam 9y ago
    Not really though. Equating docker with OCI is kind of like saying gmail is the same thing as SMTP. Gmail works with SMTP to send messages, and docker works within the confines of OCI to create images. While Docker is mostly OCI compatible these days (I say mostly as they extend their tools beyond it, making it hard to replicate the build without using docker itself. Not in-and-of itself a bad thing, but still annoying for portability), it shouldn't be confused with the OCI itself, which is there to help safeguard against companies like Docker going off and doing incompatible things.
- xxxn 9y ago
  rkt's roadmap is to move from its own aci spec to oci as soon as the latter is "ready" maybe i have the wording of this wrong, rkt people please advise
  - philips 9y ago
    rkt implements and can pull OCI containers today. appc is deprecated.
    - xxxn 9y ago
      goddamnit. ive spent the last week learning about containers via rkt, having deliberately avoided the docker hype for the last few years. ive mostly ported my setup to rkt, using acis because thats what all the rkt docs talk about. ive invested time into the 3rd party dgr build tool, which works really well and is based on acis, and cant be easily ported to oci. ive set up CI, deployment etc. now it turns out ive misunderstood the situation and im going to have to throw it all away at some point soon. this is basically why i didnt bother picking up docker before. or is it moby now? yeah. should have just stuck with openvz, vagrant and shell scripts, theyve worked fine for the best part of a decade.
      - philips 9y ago
        I am sorry if the docs were unclear. I filed an issue if you could comment on where you got off track building ACIs we will try to have someone in the community fix it up: https://github.com/rkt/rkt/issues/3745 https://github.com/rkt/rkt/issues/3745
        xxxn 9y ago
        thanks. i made a comment. i was worried i was going to go back and look at the docs and see numerous glaring advisories that I'd ignored, having opted instead to read blatantly out of date pages, but no, apart from a few vague parenthetical warnings the docs all talk about building and working with ACIs and the appc spec.
        9y ago
- shykes 9y ago
  Docker uses a separate project called containerd to implement OCI: https://github.com/containerd/containerd https://github.com/containerd/containerd Containerd itself is not at all Docker-specific, you can use it independently for anything that requires transferring and running OCI containers. Because it's bundled with Docker, it's by far the most widely installed OCI implementation, easily in the tens of millions of nodes.
- jadeklerk 9y ago
  runC is the tool that fits the OCI spec. It spawns containers on your host OS. runV is another tool that fits the OCI spec - it spawns container on hypervisors (type 2 AND 1). runC is the most commonly used - it is used by containerd (which is docker's wrapper), garden (which is cloudfoundry's wrapper), and CRI-O (which is kubernete's wrapper). rkt is another container company, whose wrapper is (afaik) just called rkt, and whose runC equivalent is systemd-nspawn, which does not meet OCI spec last time I checked.
  - erickt 9y ago
    FYI, rkt was created (and I believe still predominantly maintained) by CoreOS: https://coreos.com/rkt https://coreos.com/rkt
    - cyphar 9y ago
      It was created by CoreOS, but was recently donated to the Cloud Native Computing Foundation[1] (the same foundation that houses Kubernetes). However, I still believe the main contributors are CoreOS at the moment. [1]: https://www.cncf.io/projects/ https://www.cncf.io/projects/
- vishvananda 9y ago
  I agree that more implementations are necessary. My team released an oci-runtime implementation in rust a few weeks ago: https://github.com/oracle/railcar https://github.com/oracle/railcar We also released an oci-image builder that uses a different model from docker build: https://github.com/oracle/smith https://github.com/oracle/smith
zzzcpan 9y ago
Is there an rfc-like specification document somewhere?
- philips 9y ago
  The specs can be found here: https://github.com/opencontainers/image-spec/blob/master/spec.md https://github.com/opencontainers/image-spec/blob/master/spe... https://github.com/opencontainers/runtime-spec/blob/master/spec.md https://github.com/opencontainers/runtime-spec/blob/master/s...
cyphar 9y ago
Huge props to the other maintainers of the various OCI projects, the contributors to the OCI, and the wider community (special shout-outs to the AppC folks). It's been a very long time coming, but finally we pulled through and have hit this milestone. I'm very excited for all of the ideas we want to work on now that this milestone is out of the way, and we have a solid base to improve upon. But of course we should have a moment's rest to appreciate how far we've come. Also, here's the official press release from LF: https://www.opencontainers.org/blog/2017/07/19/oci-v1-0-bringing-containers-closer-to-standardization https://www.opencontainers.org/blog/2017/07/19/oci-v1-0-brin...
neom 9y ago
Been a hot minute since I've sat and thought about cloud. Curious how containers fit into the "serverless" (lol) movement? Also, does something like Invision straight to lambda poses an existential threat?
- spullara 9y ago
  Honestly I think that containers are a transitionary technology. Lambda and its ilk are the future of application development. Those systems will likely run on top of container based infrastructure so you can have a variety of lambda environments but not something the application dev worries about directly. Managing the full operating system stack is just not something application developers should have to worry about.
  - neom 9y ago
    Yeah, I guess that's true for app devs. I suppose you're right in that they make a lot more sense for infrastructure developers (who I ironically always forget about oops).
  - orthecreedence 9y ago
    It's funny how PHP had it right all along. Serverless/lambda -> stateless "functions" being passed arguments -> same as PHP (w/o sessions and junk). To be honest, when I jumped from PHP dev to more app-based backends years ago, I thought it was fairly crude to have a stateful server handling everything. Now everyone's starting to move back to (hosted) versions of what amounts to "bring up and tear down the entire app on each request."
    - rckclmbr 9y ago
      That's not what it is at all. Lambda functions persist, unless traffic drops, then it is killed. But if traffic is going to it, they try to be smart about reusing a hot app. PHP is even moving toward "stateful" server handling. Just look at hhvm
      - 9y ago
yoshuaw 9y ago
This is really cool - the Open Container bundle format is quite easy to grok; it's a tarball* containing a config.json file for metadata, and a root filesystem. See: https://github.com/opencontainers/runtime-spec/blob/master/bundle.md https://github.com/opencontainers/runtime-spec/blob/master/b... I'm really digging the end result of the spec - it mostly defines a bunch of conventions on top of easy formats. Kind of wish the marketing around it would be less buzz wordy, because the end result is quite straight forward. * (It's usually a tarball; nothing stopping you from sharing images over torrent for example.)
agumonkey 9y ago
ISO OSI OCI I see a pattern
clhodapp 9y ago
It is sad how much waste will occur because OCI used Docker's flat layer list instead of appc's dependency graph. This will lead to a tremendous amount of unnecessary network traffic and disk usage globally. I think this is a Betamax vs VHS moment on a much smaller scale.
- ejholmes 9y ago
  If we distribute OCI images via IPFS, problem solved.
- cpuguy83 9y ago
  Images don't need to be stored in this format on disk, the format is for transport/interchange. But out of curiosity, why do you say that a dependency graph would be better here? In terms of on disk storage and transport of layers, the layers are already content addressed.
  - dlor 9y ago
    Not the OP so I'm just speculating, but being able to "mount" content-addressable blobs into different locations in the resulting rootfs would enable more incrementality and sharing of blobs. With a flat list, the same logical set of files will have a different checksum depending on where in the final rootfs it needs to end up, right?
    - cpuguy83 9y ago
      These blobs are already shared. (In Docker) Since each layer is only the diff from the layer before it, the content address is only from the change set. Technically you can share blobs even between an image that is `FROM centos` and one that is `FROM ubuntu` as long as the layer hashes are the same. If you have a hierarchical dependency chain, and addressing content based on the content that comes before it in the chain, you can only share blobs between images if they have a common ancestry.
      - dlor 9y ago
nwmcsween 9y ago
I think a better but os specific alternative would be to simply have an elf section in a binary per 'container' technology (mounts, unshare, seccomp, etc) that is simply a called at the earliest .ctor. State would be per binary and could potentially be per application as well in say /state/$prog/$app. The upsides of this is simplicity, permanent 'containerization', better handling of state, automatic updates if all this is handled with a package manager and finally the potential to have more locked down containers via analyzing the dep chain.

217 points by philips 9y ago

virgil_disgr4ce 9y ago

What happens if I get pulled over with this spec in my car? X-)

sctb 9y ago
We've asked you already to please stop posting unsubstantively like this, and we ban accounts that refuse.
virgil_disgr4ce 9y ago
No jokes allowed I guess :)

chubot 9y ago

Does anyone have experience with non-Docker OCI implementations to share? What's an example of something that uses OCI but not Docker? The blog post is a little vague. I applaud this effort because Docker is sloppy, but I also realize that this isn't usually how technological adoption occurs. Usually the sloppy implementation is standardized, e.g. JavaScript, POSIX, CSV, Markdown, Ruby and Python to some extent -- rather than something designed from scratch or designed with fewer implementation artifacts. Maybe C++ is an exception, where things are actually standardized and there are multiple, healthy, competing implementations.

philips 9y ago
See the implementation lists for both specs: https://github.com/opencontainers/image-spec/blob/master/implementations.md https://github.com/opencontainers/image-spec/blob/master/imp... https://github.com/opencontainers/runtime-spec/blob/master/implementations.md https://github.com/opencontainers/runtime-spec/blob/master/i... Now that things are 1.0 we expect many people to start implementing the spec. For example, the transition steps required to add OCI v1.0 support on-top of Docker v2.2 support for an application primarily involves changing a few strings. See the implementation notes here. https://github.com/opencontainers/image-spec/blob/6060c3b6c6c81bdda7b113bc17299bab6633aa1c/media-types.md https://github.com/opencontainers/image-spec/blob/6060c3b6c6...
pjmlp 9y ago
> Maybe C++ is an exception, where things are actually standardized and there are multiple, healthy, competing implementations. I see you missed the 1990 and early 2000's part of C++ history.
- chubot 9y ago
  Why don't gcc and MS C++ count? I think gcc was a little behind yes, but it caught up. Also I think there were several other independent C++ vendors in the 90's, whose names I don't recall.
  - pjmlp 9y ago
    Just out of my mind, as an early C++ adopter (1993). Turbo C++, Borland C++, Watcom C++, Green Hills C++, Microsoft C++ 7.0, djgpp for MS-DOS. Turbo C++, Borland C++, Watcom C++, Green Hills C++, Microsoft C++ 7.0, C++ Builder, Symantec C++, Metrowerks C++, C++ Builder, Visual C++ for Windows 3.x onwards. MPW C++, Metrowerks C++, Visual C++ for Mac OS. Borland C++, CSet++, Visual Age for C++ for OS/2. aC++ for HP-UX, xlC++ for Aix, Sun Forte C++ for Solaris. gcc for most UNIX systems. These are just the ones I remember without having to do a search. Writing portable C++ code in those days was quite an adventure, specially since we only had the CFront followed by C++ ARM as standards, while ANSI/ISO were working on the first actual standard.
    - chubot 9y ago
      Sure, but you're not really disagreeing with me. I was saying that C++ wasn't dominated by a particular implementation, i.e. it wasn't a "fake standard". It was and is a standard that MANY independent implementations tried to conform to!
bkeroack 9y ago
> Usually the sloppy implementation is standardized That's what this is. OCI v1.0 is pretty much "what Docker does".
- cyphar 9y ago
  While this is somewhat true for the image-spec (with some very notable exceptions where the semantics have deviated heavily), it's absolutely untrue for the runtime-spec. As one of the people who maintains the legacy of what Docker did before the OCI (libcontainer which is now part of runc), the interfaces and semantics are very significantly different. Docker does use the OCI runtime-spec internally now (with plans to support the image-spec at some point), but that happened after the spec had become fairly stable. The most important thing is not where the spec design came from, but that it's now being maintained by a community that will make sure you don't end up with fragmentation and lock-in.
  - pcthrowaway 9y ago
    How much does the OCI image spec differ from the one developed by CoreOS for appc? When OCI was announced, image spec was outside of its scope (https://coreos.com/blog/making-sense-of-standards.html https://coreos.com/blog/making-sense-of-standards.html), so coreos's work in this space seemed highly important. So has all work essentially been abandoned? Or did it inform the image spec of OCI (and 'what docker does') at all?
- scaryclam 9y ago
  Not really though. Equating docker with OCI is kind of like saying gmail is the same thing as SMTP. Gmail works with SMTP to send messages, and docker works within the confines of OCI to create images. While Docker is mostly OCI compatible these days (I say mostly as they extend their tools beyond it, making it hard to replicate the build without using docker itself. Not in-and-of itself a bad thing, but still annoying for portability), it shouldn't be confused with the OCI itself, which is there to help safeguard against companies like Docker going off and doing incompatible things.
xxxn 9y ago
rkt's roadmap is to move from its own aci spec to oci as soon as the latter is "ready" maybe i have the wording of this wrong, rkt people please advise
- philips 9y ago
  rkt implements and can pull OCI containers today. appc is deprecated.
  - xxxn 9y ago
    goddamnit. ive spent the last week learning about containers via rkt, having deliberately avoided the docker hype for the last few years. ive mostly ported my setup to rkt, using acis because thats what all the rkt docs talk about. ive invested time into the 3rd party dgr build tool, which works really well and is based on acis, and cant be easily ported to oci. ive set up CI, deployment etc. now it turns out ive misunderstood the situation and im going to have to throw it all away at some point soon. this is basically why i didnt bother picking up docker before. or is it moby now? yeah. should have just stuck with openvz, vagrant and shell scripts, theyve worked fine for the best part of a decade.
    - philips 9y ago
      I am sorry if the docs were unclear. I filed an issue if you could comment on where you got off track building ACIs we will try to have someone in the community fix it up: https://github.com/rkt/rkt/issues/3745 https://github.com/rkt/rkt/issues/3745
      - xxxn 9y ago
        thanks. i made a comment. i was worried i was going to go back and look at the docs and see numerous glaring advisories that I'd ignored, having opted instead to read blatantly out of date pages, but no, apart from a few vague parenthetical warnings the docs all talk about building and working with ACIs and the appc spec.
      - 9y ago
shykes 9y ago
Docker uses a separate project called containerd to implement OCI: https://github.com/containerd/containerd https://github.com/containerd/containerd Containerd itself is not at all Docker-specific, you can use it independently for anything that requires transferring and running OCI containers. Because it's bundled with Docker, it's by far the most widely installed OCI implementation, easily in the tens of millions of nodes.
jadeklerk 9y ago
runC is the tool that fits the OCI spec. It spawns containers on your host OS. runV is another tool that fits the OCI spec - it spawns container on hypervisors (type 2 AND 1). runC is the most commonly used - it is used by containerd (which is docker's wrapper), garden (which is cloudfoundry's wrapper), and CRI-O (which is kubernete's wrapper). rkt is another container company, whose wrapper is (afaik) just called rkt, and whose runC equivalent is systemd-nspawn, which does not meet OCI spec last time I checked.
- erickt 9y ago
  FYI, rkt was created (and I believe still predominantly maintained) by CoreOS: https://coreos.com/rkt https://coreos.com/rkt
  - cyphar 9y ago
    It was created by CoreOS, but was recently donated to the Cloud Native Computing Foundation[1] (the same foundation that houses Kubernetes). However, I still believe the main contributors are CoreOS at the moment. [1]: https://www.cncf.io/projects/ https://www.cncf.io/projects/
vishvananda 9y ago
I agree that more implementations are necessary. My team released an oci-runtime implementation in rust a few weeks ago: https://github.com/oracle/railcar https://github.com/oracle/railcar We also released an oci-image builder that uses a different model from docker build: https://github.com/oracle/smith https://github.com/oracle/smith

zzzcpan 9y ago

Is there an rfc-like specification document somewhere?

philips 9y ago
The specs can be found here: https://github.com/opencontainers/image-spec/blob/master/spec.md https://github.com/opencontainers/image-spec/blob/master/spe... https://github.com/opencontainers/runtime-spec/blob/master/spec.md https://github.com/opencontainers/runtime-spec/blob/master/s...

cyphar 9y ago

Huge props to the other maintainers of the various OCI projects, the contributors to the OCI, and the wider community (special shout-outs to the AppC folks). It's been a very long time coming, but finally we pulled through and have hit this milestone. I'm very excited for all of the ideas we want to work on now that this milestone is out of the way, and we have a solid base to improve upon. But of course we should have a moment's rest to appreciate how far we've come. Also, here's the official press release from LF: https://www.opencontainers.org/blog/2017/07/19/oci-v1-0-bringing-containers-closer-to-standardization https://www.opencontainers.org/blog/2017/07/19/oci-v1-0-brin...

neom 9y ago

Been a hot minute since I've sat and thought about cloud. Curious how containers fit into the "serverless" (lol) movement? Also, does something like Invision straight to lambda poses an existential threat?

spullara 9y ago
Honestly I think that containers are a transitionary technology. Lambda and its ilk are the future of application development. Those systems will likely run on top of container based infrastructure so you can have a variety of lambda environments but not something the application dev worries about directly. Managing the full operating system stack is just not something application developers should have to worry about.
- neom 9y ago
  Yeah, I guess that's true for app devs. I suppose you're right in that they make a lot more sense for infrastructure developers (who I ironically always forget about oops).
- orthecreedence 9y ago
  It's funny how PHP had it right all along. Serverless/lambda -> stateless "functions" being passed arguments -> same as PHP (w/o sessions and junk). To be honest, when I jumped from PHP dev to more app-based backends years ago, I thought it was fairly crude to have a stateful server handling everything. Now everyone's starting to move back to (hosted) versions of what amounts to "bring up and tear down the entire app on each request."
  - rckclmbr 9y ago
    That's not what it is at all. Lambda functions persist, unless traffic drops, then it is killed. But if traffic is going to it, they try to be smart about reusing a hot app. PHP is even moving toward "stateful" server handling. Just look at hhvm
    - 9y ago

yoshuaw 9y ago

This is really cool - the Open Container bundle format is quite easy to grok; it's a tarball* containing a config.json file for metadata, and a root filesystem. See: https://github.com/opencontainers/runtime-spec/blob/master/bundle.md https://github.com/opencontainers/runtime-spec/blob/master/b... I'm really digging the end result of the spec - it mostly defines a bunch of conventions on top of easy formats. Kind of wish the marketing around it would be less buzz wordy, because the end result is quite straight forward. * (It's usually a tarball; nothing stopping you from sharing images over torrent for example.)

agumonkey 9y ago

ISO OSI OCI I see a pattern

clhodapp 9y ago

It is sad how much waste will occur because OCI used Docker's flat layer list instead of appc's dependency graph. This will lead to a tremendous amount of unnecessary network traffic and disk usage globally. I think this is a Betamax vs VHS moment on a much smaller scale.

ejholmes 9y ago
If we distribute OCI images via IPFS, problem solved.
cpuguy83 9y ago
Images don't need to be stored in this format on disk, the format is for transport/interchange. But out of curiosity, why do you say that a dependency graph would be better here? In terms of on disk storage and transport of layers, the layers are already content addressed.
- dlor 9y ago
  Not the OP so I'm just speculating, but being able to "mount" content-addressable blobs into different locations in the resulting rootfs would enable more incrementality and sharing of blobs. With a flat list, the same logical set of files will have a different checksum depending on where in the final rootfs it needs to end up, right?
  - cpuguy83 9y ago
    These blobs are already shared. (In Docker) Since each layer is only the diff from the layer before it, the content address is only from the change set. Technically you can share blobs even between an image that is `FROM centos` and one that is `FROM ubuntu` as long as the layer hashes are the same. If you have a hierarchical dependency chain, and addressing content based on the content that comes before it in the chain, you can only share blobs between images if they have a common ancestry.
    - dlor 9y ago

nwmcsween 9y ago

I think a better but os specific alternative would be to simply have an elf section in a binary per 'container' technology (mounts, unshare, seccomp, etc) that is simply a called at the earliest .ctor. State would be per binary and could potentially be per application as well in say /state/$prog/$app. The upsides of this is simplicity, permanent 'containerization', better handling of state, automatic updates if all this is handled with a package manager and finally the potential to have more locked down containers via analyzing the dep chain.