5 ms·

The "Use compose-files to deploy swarm mode services" is really intriguing to me, especially when they quote this one liner: > docker stack deploy --compose-fi

by dchuk 10y ago

The "Use compose-files to deploy swarm mode services" is really intriguing to me, especially when they quote this one liner: > docker stack deploy --compose-file=docker-compose.yml my_stack But there's no links to further documentation or anything. Can this be used to deploy easily to a cluster of Droplets for instance? I feel like the low end/longtail deployment of Docker is really underserved. I want to use Docker for its devops merits, but I have yet to find a clear, concise guide for deploying a simple web app to one or a cluster of VPS instances for a modest traffic project.

andrewstuart2 10y ago
Have you looked at Kubernetes? It's pretty dead simple to get a cluster up these days and the general abstractions they chose are really great. Service discovery, config and secret storage, control loops, autoscaling, even stateful containers these days. 100% worth giving it a shot, even just to say you've tried it. I've been using Docker for over three years now, and Kubernetes really is the realization of what I imagined containers would be like when I was just starting to use them in development.
- dchuk 10y ago
  I haven't yet. Any good getting started guides/tutorials you could recommend?
  - alpb 10y ago
    I suggest jumping right into the documentation on kuberenetes.io. There is got decent documentation with a set of tutorials and you can check out this playground with a walkthrough: https://www.katacoda.com/courses/kubernetes/playground https://www.katacoda.com/courses/kubernetes/playground
- sandGorgon 10y ago
  service discovery is not yet in kubernetes. the proposal is here - https://github.com/kubernetes/community/blob/master/contributors/design-proposals/service-discovery.md https://github.com/kubernetes/community/blob/master/contribu... and https://github.com/kubernetes/community/blob/a1d8453e184dfd8b1c5656e1e42528125d2eb906/contributors/design-proposals/aggregated-api-servers.md https://github.com/kubernetes/community/blob/a1d8453e184dfd8... secret storage is not done - this is the bug tracking the Hashicorp Vault proposal https://github.com/kubernetes/kubernetes/issues/10439 https://github.com/kubernetes/kubernetes/issues/10439 The configuration scheme is not fixed yet - https://github.com/kubernetes/kubernetes/issues/10439 https://github.com/kubernetes/kubernetes/issues/10439 The game is still up in the air. While I'm bullish on Kubernetes, its still not the superior choice.
  - TheDong 10y ago
    Your response is a massive mischaractarization of the status of kubernetes. It's FUD. 1. Service discovery. When people say service discovery, they usually mean the ability for a given application to discover other copies of itself and other services & applications within a cluster. Kubernetes has this down. It has it down better than almost any other system. The "Service" object in the API can be used for in-cluster service discovery, the service-account tokens can be used as a powerful means of introspection and discovery, etc. The things you link to are neither relevant. The first is about making api-servers more HA and federation better I think (unrelated to user's services) and the second is a proposal which is implemented, done, and didn't really catch on tbh. It's implemented because, well, it didn't propose any code changes on top of all the features services provide now, just some standard metadata users can opt in to adding if they want that sorta thing. 2. Secret storage The Secret api object works great. It's done. Adding vualt support is an ongoing feature, but it is in no way a bug, it's just a feature request/enhancement. It's good that kubernetes is evolving, but that doesn't mean that the feature isn't already working. Configuration scheme You linked the same issue as before, typo I assume. I have no clue what you're talking about though. ConfigMaps are basically done. Downward API is nice. No clue what you think isn't "fixed". Please quit it with your FUD. You obviously have no clue wtf you're talking about
    - sandGorgon 10y ago
      not only are you being very liberal with your use of profanity, you are not very accurate. I am not sure what is your personal peeve on this that you had to use words which have no place in a technical conversation. I linked to bugs and proposals that we are discussing on various sigs on slack. even aspects of load balancers and Service abstraction are insufficient and im part of some of the discussions. secrets management being insufficient is the reason why there are suitable number of distributions with their own flavors of these. Your terming of Service Discovery proposal as a HA is laughable. HA in apiservers has been production ready for quite some time and well integrated in kops and kargo. Etcd recovery is still a hassle. You are mistaking documentation being marked as available with it being actually usable in lifecycle. Are you aware that full tls in kubernetes is hard because some values are hardcoded? this causes etcd breakage in lifecycles. kubernetes is not complete - im very bullish on its future, but I would recommend you stick to technical refutal more than personal ire.
- ekidd 10y ago
  > It's pretty dead simple to get a cluster up these days The standard Kubernetes setup instructions are horrible. You need to run a bunch of shell scripts including cluster/kube-up.sh, which are documented to work on AWS, but which are completely untested before release, and which were recently broken for almost a month. I'm currently running Kubernetes under Rancher. It's still a pretty steep learning curve, with some very well-hidden but essential configuration parameters, but at least it actually works if you follow the instructions.
  - lentil 10y ago
    The situation has been improved quite a bit with kubeadm. Getting a basic cluster set up is a lot easier with that. https://kubernetes.io/docs/getting-started-guides/kubeadm/ https://kubernetes.io/docs/getting-started-guides/kubeadm/
  - ecliptik 10y ago
    I've found Rancher to best way to setup a k8s cluster as well. Even the "Kubernetes of Children" book made my head spin.
nhumrich 10y ago
I would try out rancher. It's kinda like kubernetes but uses docker lingo. Kubernetes has a somewhat high learning curve (what's a replication controller? Why is that different from a pod?). Rancher will even spin up new hosts for you on DO/AWS/anythihg and put them in the cluster.
- ecliptik 10y ago
  They even have a demo site now: https://try.rancher.com https://try.rancher.com
RRRA 10y ago
And my understanding is that all the capabilities and security options that were in 2.1 version are not supported yet... :(
fapjacks 10y ago
There is also Docker Cloud (and Docker Datacenter for bigger fish). You can bring your own nodes of whatever, e.g. AWS EC2, DO, bare metal somewhere. The documentation for Cloud tackles this kind of thing.
nzoschke 10y ago
If you want to deploy to AWS and ECS check out: Convox - https://github.com/convox/rack https://github.com/convox/rack ECS CLI - http://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-service.html http://docs.aws.amazon.com/AmazonECS/latest/developerguide/c... Disclaimer: I work at Convox
nzoschke 10y ago
If you want to deploy to AWS and ECS check out: Convox - https://github.com/convox/rack https://github.com/convox/rack ECS CLI - http://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-service.html http://docs.aws.amazon.com/AmazonECS/latest/developerguide/c... Disclaimer: I work at Convox
zenlikethat 10y ago
Since no one has actually provided links to more info about Compose V3, I'm happy to step in and help. > But there's no links to further documentation or anything. Can this be used to deploy easily to a cluster of Droplets for instance? Yes! This is exactly what the Compose V3 format (now native to Docker CLI, as noted in the article) is intended for. It creates/manages native Docker swarm services (and networks, etc.). The Docker official docs might take a minute to catch up in terms of Google indexing, etc., but you can always get the latest on GitHub to get a feel for what has changed from previous versions and/or how to learn Compose V3. https://github.com/docker/docker.github.io/blob/master/compose/compose-file.md https://github.com/docker/docker.github.io/blob/master/compo... is a Markdown document outlining various Compose options. The main thing to keep in mind is that 'build' will not work as it has traditionally (use 'image' instead), and you gain additional access to a 'deploy' key to specify, e.g., number of "replicas" (indentical copies for scale) of a given container. https://github.com/nathanleclaire/composeexample/blob/master/docker-compose.yml https://github.com/nathanleclaire/composeexample/blob/master... is a V3 version of the canonical Compose Redis + Python "page counter" app that may help to get the feel for things. In general: - 'docker deploy' will create or update a 'docker stack' (group of services) ('docker deploy' is shorthand/porcelain for 'docker stack deploy') - 'docker stack' will allow you to manage these stacks (list, remove, etc.), created from Compose files - 'docker service' will allow you to manage the individual services created (a service is a group of homogenous containers -- think 3 copies of a webapp intended to sit behind a load balancer) Check out the '--help' text for each.

by dchuk 10y ago

andrewstuart2 10y ago

Have you looked at Kubernetes? It's pretty dead simple to get a cluster up these days and the general abstractions they chose are really great. Service discovery, config and secret storage, control loops, autoscaling, even stateful containers these days. 100% worth giving it a shot, even just to say you've tried it. I've been using Docker for over three years now, and Kubernetes really is the realization of what I imagined containers would be like when I was just starting to use them in development.

dchuk 10y ago
I haven't yet. Any good getting started guides/tutorials you could recommend?
- alpb 10y ago
  I suggest jumping right into the documentation on kuberenetes.io. There is got decent documentation with a set of tutorials and you can check out this playground with a walkthrough: https://www.katacoda.com/courses/kubernetes/playground https://www.katacoda.com/courses/kubernetes/playground
sandGorgon 10y ago
service discovery is not yet in kubernetes. the proposal is here - https://github.com/kubernetes/community/blob/master/contributors/design-proposals/service-discovery.md https://github.com/kubernetes/community/blob/master/contribu... and https://github.com/kubernetes/community/blob/a1d8453e184dfd8b1c5656e1e42528125d2eb906/contributors/design-proposals/aggregated-api-servers.md https://github.com/kubernetes/community/blob/a1d8453e184dfd8... secret storage is not done - this is the bug tracking the Hashicorp Vault proposal https://github.com/kubernetes/kubernetes/issues/10439 https://github.com/kubernetes/kubernetes/issues/10439 The configuration scheme is not fixed yet - https://github.com/kubernetes/kubernetes/issues/10439 https://github.com/kubernetes/kubernetes/issues/10439 The game is still up in the air. While I'm bullish on Kubernetes, its still not the superior choice.
- TheDong 10y ago
  Your response is a massive mischaractarization of the status of kubernetes. It's FUD. 1. Service discovery. When people say service discovery, they usually mean the ability for a given application to discover other copies of itself and other services & applications within a cluster. Kubernetes has this down. It has it down better than almost any other system. The "Service" object in the API can be used for in-cluster service discovery, the service-account tokens can be used as a powerful means of introspection and discovery, etc. The things you link to are neither relevant. The first is about making api-servers more HA and federation better I think (unrelated to user's services) and the second is a proposal which is implemented, done, and didn't really catch on tbh. It's implemented because, well, it didn't propose any code changes on top of all the features services provide now, just some standard metadata users can opt in to adding if they want that sorta thing. 2. Secret storage The Secret api object works great. It's done. Adding vualt support is an ongoing feature, but it is in no way a bug, it's just a feature request/enhancement. It's good that kubernetes is evolving, but that doesn't mean that the feature isn't already working. Configuration scheme You linked the same issue as before, typo I assume. I have no clue what you're talking about though. ConfigMaps are basically done. Downward API is nice. No clue what you think isn't "fixed". Please quit it with your FUD. You obviously have no clue wtf you're talking about
  - sandGorgon 10y ago
    not only are you being very liberal with your use of profanity, you are not very accurate. I am not sure what is your personal peeve on this that you had to use words which have no place in a technical conversation. I linked to bugs and proposals that we are discussing on various sigs on slack. even aspects of load balancers and Service abstraction are insufficient and im part of some of the discussions. secrets management being insufficient is the reason why there are suitable number of distributions with their own flavors of these. Your terming of Service Discovery proposal as a HA is laughable. HA in apiservers has been production ready for quite some time and well integrated in kops and kargo. Etcd recovery is still a hassle. You are mistaking documentation being marked as available with it being actually usable in lifecycle. Are you aware that full tls in kubernetes is hard because some values are hardcoded? this causes etcd breakage in lifecycles. kubernetes is not complete - im very bullish on its future, but I would recommend you stick to technical refutal more than personal ire.
ekidd 10y ago
> It's pretty dead simple to get a cluster up these days The standard Kubernetes setup instructions are horrible. You need to run a bunch of shell scripts including cluster/kube-up.sh, which are documented to work on AWS, but which are completely untested before release, and which were recently broken for almost a month. I'm currently running Kubernetes under Rancher. It's still a pretty steep learning curve, with some very well-hidden but essential configuration parameters, but at least it actually works if you follow the instructions.
- lentil 10y ago
  The situation has been improved quite a bit with kubeadm. Getting a basic cluster set up is a lot easier with that. https://kubernetes.io/docs/getting-started-guides/kubeadm/ https://kubernetes.io/docs/getting-started-guides/kubeadm/
- ecliptik 10y ago
  I've found Rancher to best way to setup a k8s cluster as well. Even the "Kubernetes of Children" book made my head spin.

nhumrich 10y ago

I would try out rancher. It's kinda like kubernetes but uses docker lingo. Kubernetes has a somewhat high learning curve (what's a replication controller? Why is that different from a pod?). Rancher will even spin up new hosts for you on DO/AWS/anythihg and put them in the cluster.

ecliptik 10y ago
They even have a demo site now: https://try.rancher.com https://try.rancher.com

RRRA 10y ago

And my understanding is that all the capabilities and security options that were in 2.1 version are not supported yet... :(

fapjacks 10y ago

There is also Docker Cloud (and Docker Datacenter for bigger fish). You can bring your own nodes of whatever, e.g. AWS EC2, DO, bare metal somewhere. The documentation for Cloud tackles this kind of thing.

nzoschke 10y ago

If you want to deploy to AWS and ECS check out: Convox - https://github.com/convox/rack https://github.com/convox/rack ECS CLI - http://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-service.html http://docs.aws.amazon.com/AmazonECS/latest/developerguide/c... Disclaimer: I work at Convox

nzoschke 10y ago

zenlikethat 10y ago

Since no one has actually provided links to more info about Compose V3, I'm happy to step in and help. > But there's no links to further documentation or anything. Can this be used to deploy easily to a cluster of Droplets for instance? Yes! This is exactly what the Compose V3 format (now native to Docker CLI, as noted in the article) is intended for. It creates/manages native Docker swarm services (and networks, etc.). The Docker official docs might take a minute to catch up in terms of Google indexing, etc., but you can always get the latest on GitHub to get a feel for what has changed from previous versions and/or how to learn Compose V3. https://github.com/docker/docker.github.io/blob/master/compose/compose-file.md https://github.com/docker/docker.github.io/blob/master/compo... is a Markdown document outlining various Compose options. The main thing to keep in mind is that 'build' will not work as it has traditionally (use 'image' instead), and you gain additional access to a 'deploy' key to specify, e.g., number of "replicas" (indentical copies for scale) of a given container. https://github.com/nathanleclaire/composeexample/blob/master/docker-compose.yml https://github.com/nathanleclaire/composeexample/blob/master... is a V3 version of the canonical Compose Redis + Python "page counter" app that may help to get the feel for things. In general: - 'docker deploy' will create or update a 'docker stack' (group of services) ('docker deploy' is shorthand/porcelain for 'docker stack deploy') - 'docker stack' will allow you to manage these stacks (list, remove, etc.), created from Compose files - 'docker service' will allow you to manage the individual services created (a service is a group of homogenous containers -- think 3 copies of a webapp intended to sit behind a load balancer) Check out the '--help' text for each.