7 ms·

Docker 1.8 released

107 points by Spiritus 11y ago

anh79 11y ago
Congratulations! It seems the memory issue with "docker" daemon hasn't been admitted/resolved. (See https://github.com/docker/docker/releases/tag/v1.8.0 https://github.com/docker/docker/releases/tag/v1.8.0) In all of my deployments, "docker" daemon memory keeps increasing. The worst case is 37% in a 512MB instance in Digital Ocean. Then docker daemon crashes... That's to say, congratulate but I don't really want to upgrade ... :(
- cpuguy83 11y ago
  Memory issues can be caused by a number of things. There's been a lot of work done to improve some leakages around the logging -- both by being able to disable logging if you don't need it or switch to a different logger, and general improvements to the default jsonlog logger. There's also been a known issue around creating lots of `docker exec` instances and those references not being released (so you can query for status later), this is also resolved. It would be awesome if you could profile it and submit a bug report with how you are using docker and where you are seeing memory growth. Let me know how I can help so we can get this resolved.
  - anh79 11y ago
    Hi @cpuguy83, Thanks for your feedback. I couldn't gather enough information for bug reporting (But I had few rich logs that told "docker" has invalid memory access.) As a work-around, I use a special patch of docker-compose that helps the containers to restart automatically when "docker" daemon restarts :) > There's also been a known issue around creating lots of `docker exec` instances and those references not being released (so you can query for status later), this is also resolved. Woh, this seems my case, though I am not sure about "releasing" (We run "exec" and "exit"; we don't hold any "exec" for longer than 2 seconds.) I haven't seen this mentioned in changelog (FIXME). Let me try again in ticket system!
    - cpuguy83 11y ago
      Yes, it's docker that was holding on to these exec instances (and cluttering up the `docker inspect` output if you do a bunch). We now do GC on these. ps, you can add `reastart: always` to your compose file, and docker daemon will automatically restart those containers for you.
      - anh79 11y ago
        Thanks a lot. I wonder why this is not mentioned in the changelog :) Regarding the `docker-compose` issue, it's described here https://github.com/docker/compose/pull/1723 https://github.com/docker/compose/pull/1723 . Aanand Prasad has a better PR in https://github.com/docker/compose/pull/1754 https://github.com/docker/compose/pull/1754 (merged).
reustle 11y ago
Glad to see they finally gave some love to running docker on osx, as it is definitely a pain point for a lot of people. Hopefully Docker Toolbox is what we've been waiting for.
- bkeroack 11y ago
  I imagine it's still using virtualization under the hood, so it's no more "real" Docker than spinning up a Vagrant ubuntu box and running containers there.
- SloopJon 11y ago
  I started looking at Docker during vacation last week. I had a slightly outdated copy of Boot2Docker that didn't play well with VirtualBox 5.0. With limited bandwidth, I downloaded the docker-machine executable rather than the whole Boot2Docker package. Worked fine, and I was able to set up a variety of containers. When I got back to work this week, I started setting some of this up on my work Mac, and found a release candidate for the Docker Toolbox. It looks like it's the docker, docker-machine, and docker-compose executables plus the Kitematic GUI (haven't tried that yet). Looks like a decent package. Grumpy, off-topic digression after a week with limited Internet: please supply offline docs (Unreal), don't put a git clone command in the configure script to build them (Rust), don't require an external crate in a tutorial (random number generator for Rust), don't put a link to a CDN in your offline docs (Leap Motion SDK), don't require activation after a 2 GB download (Unity Personal). I did luck out and find a WebSocket library in the Go tutorial, which I used to grab JSON from the Leap to load into MongoDB (in a Docker container). Grumpy, mostly on-topic digression: docker-compose doesn't work on my work Mac, and no one seems to know how to fix it: https://github.com/docker/compose/issues/271 https://github.com/docker/compose/issues/271 Haven't yet managed to rebuild it locally as suggested. Edit: some observations after installing Docker Toolbox. Kitematic is an Electron app (complete with a Toggle DevTools menu that covers the window with a web dev console). It doesn't tell you what it's doing, but docker-machine shows that it created a machine named default using the VirtualBox driver with the default settings. Opening the Docker Quickstart Terminal stopped (and deleted?) the machine that Kitematic made and started a new one with the same name and settings. I think I'd be just as happy with only the docker and docker-machine executables.
jnsaff2 11y ago
For me pushing new images to (our own) registry are failing with 1.8. Be careful.
- teabee89 11y ago
  Hey sorry to hear about that, would you mind opening an issue at https://github.com/docker/docker/issues/new https://github.com/docker/docker/issues/new with additional information (versions, logs, how to reproduce) ?
eropple 11y ago
So are the docs out of date, or does Docker still not use user namespaces? This was promised quite a while ago and has slipped multiple versions so far, and it's a lot more important to serious uses of Docker than `docker cp`. https://docs.docker.com/articles/security/ https://docs.docker.com/articles/security/
- bfirsh 11y ago
  This is still a work-in-progress. You can keep track of its status here: https://github.com/docker/docker/issues/15187 https://github.com/docker/docker/issues/15187
- omni 11y ago
  > it's a lot more important to serious uses of Docker than `docker cp` This is a cheap shot and has no basis in the reality of how open source projects are managed and how people contribute to them.
  - eropple 11y ago
    I don't see much daylight between Docker the open-source project and Docker the venture-backed company trying to convince my clients to buy into their platform and not the other venture-backed companies' platforms. The Docker company and its employees have talked a lot of game and their security story is consistently subpar, be it recommended Dockerfiles running stuff as root users or the lack of stuff like user namespaces or running as a giant daemon where even read-only operations must run as root. The literally-literally most important security concern in containerland, not allowing container escapes to have host root, is not done by Docker. But `docker cp` is, built at least in part by people working for Docker-the-company (I went and looked to be sure). So you can call it a cheap shot, and that's fine, but this is something promised a long time ago and my concern is for the people who buy the hype because they don't know any better, not the people generating it and should.
    - shykes 11y ago
      Just to clarify: user namespaces will be a great additional layer in security, but they are still a relatively new feature, with very little actual use in production. You can already use them in runC [1] (Docker's low-level container runtime which we have broken out as a standalone tool). And once we have sorted out the hairy side-effects on UID mapping and network namespaces, we will integrate it into the Docker engine itself. But even then, we won't recommend running untrusted code as root inside a Linux container until the community has more time to battle-test user namespaces. [1] https://github.com/opencontainers/runc https://github.com/opencontainers/runc
lewq 11y ago
Really happy to see volume plugins in stable :) We wrote up a demo of using volume plugins in docker 1.8 with compose here, in case people are interested: https://clusterhq.com/2015/08/04/docker-volume-plugins/ https://clusterhq.com/2015/08/04/docker-volume-plugins/
rcarmo 11y ago
Was kind of hoping they'd enable IFF_MULTICAST (https://github.com/docker/docker/issues/3043 https://github.com/docker/docker/issues/3043), but am glad to see it's slated for 1.9.0.
kiyoto 11y ago
Great to see the support for Fluentd and GELF logging drivers =) Logging is such a big part of production deployment, and it's good to see a couple of solid options in Docker's core.
spdustin 11y ago
Been using Dinghy† for my OS X Docker needs, as it sets up a relatively painless NFS share for mounting my user directory into the Docker VM. Has file sharing on Mac been improved with Docker Toolbox? VirtualBox's native file sharing is quite slow. † https://github.com/codekitchen/dinghy https://github.com/codekitchen/dinghy
- tracker1 11y ago
  I've been just running a full on Ubuntu VM, then installing SAMBA for sharing the necessary paths... so I can edit in my host OS, but run via SSH into the VM.

107 points by Spiritus 11y ago

anh79 11y ago

Congratulations! It seems the memory issue with "docker" daemon hasn't been admitted/resolved. (See https://github.com/docker/docker/releases/tag/v1.8.0 https://github.com/docker/docker/releases/tag/v1.8.0) In all of my deployments, "docker" daemon memory keeps increasing. The worst case is 37% in a 512MB instance in Digital Ocean. Then docker daemon crashes... That's to say, congratulate but I don't really want to upgrade ... :(

cpuguy83 11y ago
Memory issues can be caused by a number of things. There's been a lot of work done to improve some leakages around the logging -- both by being able to disable logging if you don't need it or switch to a different logger, and general improvements to the default jsonlog logger. There's also been a known issue around creating lots of `docker exec` instances and those references not being released (so you can query for status later), this is also resolved. It would be awesome if you could profile it and submit a bug report with how you are using docker and where you are seeing memory growth. Let me know how I can help so we can get this resolved.
- anh79 11y ago
  Hi @cpuguy83, Thanks for your feedback. I couldn't gather enough information for bug reporting (But I had few rich logs that told "docker" has invalid memory access.) As a work-around, I use a special patch of docker-compose that helps the containers to restart automatically when "docker" daemon restarts :) > There's also been a known issue around creating lots of `docker exec` instances and those references not being released (so you can query for status later), this is also resolved. Woh, this seems my case, though I am not sure about "releasing" (We run "exec" and "exit"; we don't hold any "exec" for longer than 2 seconds.) I haven't seen this mentioned in changelog (FIXME). Let me try again in ticket system!
  - cpuguy83 11y ago
    Yes, it's docker that was holding on to these exec instances (and cluttering up the `docker inspect` output if you do a bunch). We now do GC on these. ps, you can add `reastart: always` to your compose file, and docker daemon will automatically restart those containers for you.
    - anh79 11y ago
      Thanks a lot. I wonder why this is not mentioned in the changelog :) Regarding the `docker-compose` issue, it's described here https://github.com/docker/compose/pull/1723 https://github.com/docker/compose/pull/1723 . Aanand Prasad has a better PR in https://github.com/docker/compose/pull/1754 https://github.com/docker/compose/pull/1754 (merged).

reustle 11y ago

Glad to see they finally gave some love to running docker on osx, as it is definitely a pain point for a lot of people. Hopefully Docker Toolbox is what we've been waiting for.

bkeroack 11y ago
I imagine it's still using virtualization under the hood, so it's no more "real" Docker than spinning up a Vagrant ubuntu box and running containers there.
SloopJon 11y ago
I started looking at Docker during vacation last week. I had a slightly outdated copy of Boot2Docker that didn't play well with VirtualBox 5.0. With limited bandwidth, I downloaded the docker-machine executable rather than the whole Boot2Docker package. Worked fine, and I was able to set up a variety of containers. When I got back to work this week, I started setting some of this up on my work Mac, and found a release candidate for the Docker Toolbox. It looks like it's the docker, docker-machine, and docker-compose executables plus the Kitematic GUI (haven't tried that yet). Looks like a decent package. Grumpy, off-topic digression after a week with limited Internet: please supply offline docs (Unreal), don't put a git clone command in the configure script to build them (Rust), don't require an external crate in a tutorial (random number generator for Rust), don't put a link to a CDN in your offline docs (Leap Motion SDK), don't require activation after a 2 GB download (Unity Personal). I did luck out and find a WebSocket library in the Go tutorial, which I used to grab JSON from the Leap to load into MongoDB (in a Docker container). Grumpy, mostly on-topic digression: docker-compose doesn't work on my work Mac, and no one seems to know how to fix it: https://github.com/docker/compose/issues/271 https://github.com/docker/compose/issues/271 Haven't yet managed to rebuild it locally as suggested. Edit: some observations after installing Docker Toolbox. Kitematic is an Electron app (complete with a Toggle DevTools menu that covers the window with a web dev console). It doesn't tell you what it's doing, but docker-machine shows that it created a machine named default using the VirtualBox driver with the default settings. Opening the Docker Quickstart Terminal stopped (and deleted?) the machine that Kitematic made and started a new one with the same name and settings. I think I'd be just as happy with only the docker and docker-machine executables.

jnsaff2 11y ago

For me pushing new images to (our own) registry are failing with 1.8. Be careful.

teabee89 11y ago
Hey sorry to hear about that, would you mind opening an issue at https://github.com/docker/docker/issues/new https://github.com/docker/docker/issues/new with additional information (versions, logs, how to reproduce) ?

eropple 11y ago

So are the docs out of date, or does Docker still not use user namespaces? This was promised quite a while ago and has slipped multiple versions so far, and it's a lot more important to serious uses of Docker than `docker cp`. https://docs.docker.com/articles/security/ https://docs.docker.com/articles/security/

bfirsh 11y ago
This is still a work-in-progress. You can keep track of its status here: https://github.com/docker/docker/issues/15187 https://github.com/docker/docker/issues/15187
omni 11y ago
> it's a lot more important to serious uses of Docker than `docker cp` This is a cheap shot and has no basis in the reality of how open source projects are managed and how people contribute to them.
- eropple 11y ago
  I don't see much daylight between Docker the open-source project and Docker the venture-backed company trying to convince my clients to buy into their platform and not the other venture-backed companies' platforms. The Docker company and its employees have talked a lot of game and their security story is consistently subpar, be it recommended Dockerfiles running stuff as root users or the lack of stuff like user namespaces or running as a giant daemon where even read-only operations must run as root. The literally-literally most important security concern in containerland, not allowing container escapes to have host root, is not done by Docker. But `docker cp` is, built at least in part by people working for Docker-the-company (I went and looked to be sure). So you can call it a cheap shot, and that's fine, but this is something promised a long time ago and my concern is for the people who buy the hype because they don't know any better, not the people generating it and should.
  - shykes 11y ago
    Just to clarify: user namespaces will be a great additional layer in security, but they are still a relatively new feature, with very little actual use in production. You can already use them in runC [1] (Docker's low-level container runtime which we have broken out as a standalone tool). And once we have sorted out the hairy side-effects on UID mapping and network namespaces, we will integrate it into the Docker engine itself. But even then, we won't recommend running untrusted code as root inside a Linux container until the community has more time to battle-test user namespaces. [1] https://github.com/opencontainers/runc https://github.com/opencontainers/runc

lewq 11y ago

Really happy to see volume plugins in stable :) We wrote up a demo of using volume plugins in docker 1.8 with compose here, in case people are interested: https://clusterhq.com/2015/08/04/docker-volume-plugins/ https://clusterhq.com/2015/08/04/docker-volume-plugins/

rcarmo 11y ago

Was kind of hoping they'd enable IFF_MULTICAST (https://github.com/docker/docker/issues/3043 https://github.com/docker/docker/issues/3043), but am glad to see it's slated for 1.9.0.

kiyoto 11y ago

Great to see the support for Fluentd and GELF logging drivers =) Logging is such a big part of production deployment, and it's good to see a couple of solid options in Docker's core.

spdustin 11y ago

Been using Dinghy† for my OS X Docker needs, as it sets up a relatively painless NFS share for mounting my user directory into the Docker VM. Has file sharing on Mac been improved with Docker Toolbox? VirtualBox's native file sharing is quite slow. † https://github.com/codekitchen/dinghy https://github.com/codekitchen/dinghy

tracker1 11y ago
I've been just running a full on Ubuntu VM, then installing SAMBA for sharing the necessary paths... so I can edit in my host OS, but run via SSH into the VM.