4 ms·

I am wondering why you have a 1:1 mapping between Docker containers and processes? I have not used Docker but my impression of it was a good way to group a few

I am wondering why you have a 1:1 mapping between Docker containers and processes? I have not used Docker but my impression of it was a good way to group a few inter-related processes together while separating them from unrelated processes. Our thought was basically 1 container per role/service in our platform (app servers, workers, DB, logging, etc). If you have 1 process per container, isn't it just a bit stronger form of sandboxing a process?

marceldegraaf 13y ago
That's a fair point. In this case I was especially interested in running multiple instances of the same application on a CoreOS mini-cluster, demonstrating fleet's capabilities of scaling/distributing the containers over the cluster. However, in a real life application I think I would still keep the containers as "single-process" as possible; in our case that would mean: a single Rails application or a group of Resque workers. It would make the containers quick to boot and would increase security by sandboxing all the separate parts of a system in their own containers. I do think this is up to personal preferences, though :-).
justincormack 13y ago
The process is the unit in which things fail in an OS, so a collection of processes needs to do its own failure handling. Part of the point of these types of system is to do things like failure handling, rather than everything having to do its own.
danudey 13y ago
You can do it in several different ways, and it really depends on what you're trying to accomplish. If you were trying to build wordpress-as-a-service, for example, you might create a Docker instance which includes nginx, php-fpm, memcached, and maybe even mysql (which makes it easy to make snapshots of the entire state of the system). That said, fleet has a really interesting way of making interconnected systems, so you can create a memcached instance, an nginx instance, and a php-fpm-running-wordpress instance, and you can manage them separately. This makes it easy to upgrade memcached without touching nginx or php-fpm, or to change the nginx config without having to affect the running application server (and risk flushing the APC cache), etc. It also allows you to make everything pluggable in your design. You might want to add varnish in front of your nginx instance, for example, but sometimes you might not. That sort of flexibility is really useful in some circumstances. In our infrastructure at work, the design I would use would probably be similar to this; we have front-end services (nginx), internal services that they proxy to, internal services that those services access, asynchronous processing services, dispatch services, activemq, redis, memcached, mysql, etc. Having the flexibility to add more of any one of those things without having to scale any of the rest is really important. For other circumstances, it makes a lot more sense to bundle everything into a faux-VM. Run supervisord to launch ssh, nginx, php-fpm, memcached, and whatever else needs running. Share volumes to local filesystems so that configs, local changes, etc. can be taken care of, and you're good to go. So in the end, the real question is what do you want out of your containers? Flexibility or simplicity?
outworlder 13y ago
Either way can work. Some time ago, it was difficult to share data and networking between containers. Right now, it is so easy that I even have (without CoreOS) a Hubot in a container, with its Redis brain in another container. Hubot can talk to Redis just fine, and that's the only container in that machine which can do so. Maintaining the data storage backend gets easier this way. The Redis data is stored in a volume.

marceldegraaf 13y ago

That's a fair point. In this case I was especially interested in running multiple instances of the same application on a CoreOS mini-cluster, demonstrating fleet's capabilities of scaling/distributing the containers over the cluster. However, in a real life application I think I would still keep the containers as "single-process" as possible; in our case that would mean: a single Rails application or a group of Resque workers. It would make the containers quick to boot and would increase security by sandboxing all the separate parts of a system in their own containers. I do think this is up to personal preferences, though :-).

justincormack 13y ago

The process is the unit in which things fail in an OS, so a collection of processes needs to do its own failure handling. Part of the point of these types of system is to do things like failure handling, rather than everything having to do its own.

danudey 13y ago

You can do it in several different ways, and it really depends on what you're trying to accomplish. If you were trying to build wordpress-as-a-service, for example, you might create a Docker instance which includes nginx, php-fpm, memcached, and maybe even mysql (which makes it easy to make snapshots of the entire state of the system). That said, fleet has a really interesting way of making interconnected systems, so you can create a memcached instance, an nginx instance, and a php-fpm-running-wordpress instance, and you can manage them separately. This makes it easy to upgrade memcached without touching nginx or php-fpm, or to change the nginx config without having to affect the running application server (and risk flushing the APC cache), etc. It also allows you to make everything pluggable in your design. You might want to add varnish in front of your nginx instance, for example, but sometimes you might not. That sort of flexibility is really useful in some circumstances. In our infrastructure at work, the design I would use would probably be similar to this; we have front-end services (nginx), internal services that they proxy to, internal services that those services access, asynchronous processing services, dispatch services, activemq, redis, memcached, mysql, etc. Having the flexibility to add more of any one of those things without having to scale any of the rest is really important. For other circumstances, it makes a lot more sense to bundle everything into a faux-VM. Run supervisord to launch ssh, nginx, php-fpm, memcached, and whatever else needs running. Share volumes to local filesystems so that configs, local changes, etc. can be taken care of, and you're good to go. So in the end, the real question is what do you want out of your containers? Flexibility or simplicity?

outworlder 13y ago

Either way can work. Some time ago, it was difficult to share data and networking between containers. Right now, it is so easy that I even have (without CoreOS) a Hubot in a container, with its Redis brain in another container. Hubot can talk to Redis just fine, and that's the only container in that machine which can do so. Maintaining the data storage backend gets easier this way. The Redis data is stored in a volume.