3 ms·

The idea is that you build the image and push it to a registry. The service hosts then only pull the layers they don't already have. In practice, running a pri

by xnxn 12y ago

The idea is that you build the image and push it to a registry. The service hosts then only pull the layers they don't already have. In practice, running a private registry is a pain (last I checked the official Docker image for it crashed on startup). I like what Rocket is doing here with filesets and plain old URLs.

nickstinemates 12y ago
Specifically about filesets - take a look at the docker import command. It will take an arbitrary rootfs (tar file) and turn it in to a docker image.
- bmurphy1976 12y ago
  I've been down this path, far down this path. Docker can import arbitrary layers from a tar file either via the command line or the api. The problem is there is no official way of getting a set of arbitrary layers. That might not seem like a big deal, but when your image has something heavy like mono or java and pushes upwards of a gig or more running on a relatively puny cloud instance with poor I/O that adds up. If you want to have a much more efficient workflow, you have to roll this yourself like we did by going direct to the file system (at least for export). This is a messy pain in the ass and I would not expect most people to do it. I would be very happy if Docker stopped trying to shove the registry down our throats and gave us a model where we could substitute our own push/pull code that better utilized our existing infrastructure. This is a case where Docker feels more monolithic than it needs to be and I would be happier if it was broken up into a set of smaller more independent tools (e.g. docker, docker-push, and docker-pull).
  - nickstinemates 12y ago
    Would it surprise you to know we completely agree in theory that push and pull should not be reliant or used at all with the registry if a user so chooses? But what we do care about is when you docker pull or docker push you have a set of assumptions that are always true? Leading questions, I know, but I couldn't agree with you more and I know I'm not alone. If you don't know the truth - open source works a lot like other software projects. You gather feedback in as many different forums as possible make a guess at how to solve that, work with your developer communities and vested parties to come up with a solution iterate a bunch of times and hopefully get something out in users hands that doesn't completely suck. So the question is - Who's going to work on what and in what order? The feedback is important and with enough it would get higher on the list. But someone has to make it happen. Or make a proposal on how it would work. That would be truly welcome. Let me know how I personally, and we collectively, can help.
    - bmurphy1976 12y ago
      Trust me, I know how it works. I had a pull request to do at least one part of this but it was never accepted. I'm not bitter about it, I don't care, it wasn't right for Docker at the time. We had to get something done to meet our commitments so we hacked it and moved on. Agreement in "theory" doesn't help when you have real commitments and limited resources. > So the question is - Who's going to work on what and in what order? The feedback is important and with enough it would get higher on the list. But someone has to make it happen. Or make a proposal on how it would work. That would be truly welcome. Let me know how I personally, and we collectively, can help. The community clearly desires a more stable and pluggable core, doesn't like the registry workflow, and desires a daemonless mode. If the discussion over the last few days hasn't woken the Docker team up to that fact, then I don't know what to say. The new features are big and flashy so they get the big announcements, however, you should not be surprised that when a new announcement is dropped the community collectively responds "what about x y and z?" I think a Roadmap to 2.0 is what is missing and would give the community the confidence that is currently lacking.

by xnxn 12y ago

nickstinemates 12y ago

Specifically about filesets - take a look at the docker import command. It will take an arbitrary rootfs (tar file) and turn it in to a docker image.

bmurphy1976 12y ago
I've been down this path, far down this path. Docker can import arbitrary layers from a tar file either via the command line or the api. The problem is there is no official way of getting a set of arbitrary layers. That might not seem like a big deal, but when your image has something heavy like mono or java and pushes upwards of a gig or more running on a relatively puny cloud instance with poor I/O that adds up. If you want to have a much more efficient workflow, you have to roll this yourself like we did by going direct to the file system (at least for export). This is a messy pain in the ass and I would not expect most people to do it. I would be very happy if Docker stopped trying to shove the registry down our throats and gave us a model where we could substitute our own push/pull code that better utilized our existing infrastructure. This is a case where Docker feels more monolithic than it needs to be and I would be happier if it was broken up into a set of smaller more independent tools (e.g. docker, docker-push, and docker-pull).
- nickstinemates 12y ago
  Would it surprise you to know we completely agree in theory that push and pull should not be reliant or used at all with the registry if a user so chooses? But what we do care about is when you docker pull or docker push you have a set of assumptions that are always true? Leading questions, I know, but I couldn't agree with you more and I know I'm not alone. If you don't know the truth - open source works a lot like other software projects. You gather feedback in as many different forums as possible make a guess at how to solve that, work with your developer communities and vested parties to come up with a solution iterate a bunch of times and hopefully get something out in users hands that doesn't completely suck. So the question is - Who's going to work on what and in what order? The feedback is important and with enough it would get higher on the list. But someone has to make it happen. Or make a proposal on how it would work. That would be truly welcome. Let me know how I personally, and we collectively, can help.
  - bmurphy1976 12y ago
    Trust me, I know how it works. I had a pull request to do at least one part of this but it was never accepted. I'm not bitter about it, I don't care, it wasn't right for Docker at the time. We had to get something done to meet our commitments so we hacked it and moved on. Agreement in "theory" doesn't help when you have real commitments and limited resources. > So the question is - Who's going to work on what and in what order? The feedback is important and with enough it would get higher on the list. But someone has to make it happen. Or make a proposal on how it would work. That would be truly welcome. Let me know how I personally, and we collectively, can help. The community clearly desires a more stable and pluggable core, doesn't like the registry workflow, and desires a daemonless mode. If the discussion over the last few days hasn't woken the Docker team up to that fact, then I don't know what to say. The new features are big and flashy so they get the big announcements, however, you should not be surprised that when a new announcement is dropped the community collectively responds "what about x y and z?" I think a Roadmap to 2.0 is what is missing and would give the community the confidence that is currently lacking.