24 ms·

> ECS is really the only way I'd go with Docker ECS is AWS-specific, which is perfectly fine for some. But Kubernetes has been amazing for us. It abstracts man

by gtaylor 11y ago

> ECS is really the only way I'd go with Docker ECS is AWS-specific, which is perfectly fine for some. But Kubernetes has been amazing for us. It abstracts many of the differences between AWS/Google Cloud, it's open source, and is far more powerful and flexible. The only issue right now is that setting the cluster up involves running some shell scripts (yuck). We use Google Container Engine (hosted Kubernetes on Google Cloud), so we don't have to deal with that, but the option is there should we ever need to go multi-cloud. Figured I'd toss that out there for anyone struggling with ECS (it can be a bit rigid) or keeping an eye on things beyond AWS. Kubernetes is still young and rough in areas, but it is a nice, opinionated way to orchestrate containers.

merb 11y ago
wow so instead of running my configuration management stuff I know need to either: - be on aws - be on gcloud - any other cloud that has docker support out of the box OR: I need docker + a configuration management to setup my environment, or I need to somehow manage my coreos configs. Oh and finally don't forget to run a network over a network cause it's dockerish so for most clouds this means we run a network on a network which runs a network. docker adds so much complexity. people just don't see this right upfront and use most of their time into these stuff, but there are easier ways to deploy. YES if you are really big and if your servers needs to scale way beyond the most than you need it probably since configuration management won't help you and setting up servers even in the cloud take some time looking at some netflix articles. however i just don't get it why people use their time for docker when there are other things to do in their programs.
- pbecotte 11y ago
  But it doesn't really, it is just another tool to learn like your configuration management software. Honestly, learning Docker to a high level was WAY easier than getting my Chef knowledge to a mediocre level. Further, it has other benefits like being MUCH easier to set up well isolated dev environments with Docker than with Chef. Since we switched we can usually have someone running the dev environment and tests within an hour or so of unboxing their laptop. With chef running the whole thing from scratch took that long by itself (after you installed all the software) and usually failed. And the idea that it is too complicated for something simple- I built a sample blogging app that you can deploy to a digital ocean or ec2 machine with like 3 or 4 commands (https://github.com/pbecotte/devblog https://github.com/pbecotte/devblog). The system we have built at work has 8 separate services between a bunch of data stores, background workers, and a couple different pieces of our app- but Docker allows you to run the entire environment on your local machine and then deploy that same setup to our cluster, without any differences to account for making all those apps run on one VM for your local environment (which can get bad when one service requires a different version of a ruby gem than another for example).
  - fpoling 11y ago
    I second that. Docker makes it very easy to separate the whole application into a bunch of isolated services and then reproduce the whole production environment on a laptop for a very convenient development.
  - merb 11y ago
    How do you do the same thing without a cloud Environment? Do you install the software manually? Do you configure your network manually? Do you configure your os manually? Do you install the docker daemon manually? What about installations behind firewalls? Or about code that shouldn't belong to the docker registry since it shouldn't be pushed over the internet ? etc... Why do you have 8 separted services anyway? How many people does your company have? for 8 services you should at least have 8 * 3 people.
- gtaylor 11y ago
  > wow so instead of running my configuration management stuff I know need to either: - be on aws - be on gcloud - any other cloud that has docker support out of the box No. You need a networked computer with a quasi-recent kernel. There are no requirements to use a cloud provider or to use AWS/GCP. > OR: I need docker + a configuration management to setup my environment, or I need to somehow manage my coreos configs. Oh and finally don't forget to run a network over a network cause it's dockerish so for most clouds this means we run a network on a network which runs a network. This sounds like aimless rambling. You don't have to use any of this. You can, of course, but I can do some nasty stuff without Docker as well. Like I mentioned with Container Engine, setting up a Kubernetes cluster is like two mouse clicks. You can get as simple as ECS, or you can roll your own from the ground up. You have the option to pick a point on a spectrum. With ECS, it's Amazon's way or the highway. > however i just don't get it why people use their time for docker when there are other things to do in their programs. Because it saves us loads of time developing, testing, and deploying our systems. There is more initial setup work, but after that we are deploying images seamlessly, have a great rolling upgrade and rollback story out of the box, and get a lot of other bonuses like service discovery, auto-scaling (vertically and horizontally), and much better (higher) resource usage levels. And we're far from a mega-corp. To me, it sounds like you may have skimmed some, perhaps even played a bit. But you ran into a snag, threw up your hands, and have summarily dismissed an entire ecosystem after your experience(s). I see enough un-informed or 100% incorrect things above to think you're missing some details.
  - merb 11y ago
    > Because it saves us loads of time developing, testing, and deploying our systems. There is more initial setup work, but after that we are deploying images seamlessly, have a great rolling upgrade and rollback story out of the box, and get a lot of other bonuses like service discovery, auto-scaling (vertically and horizontally), and much better (higher) resource usage levels. And we're far from a mega-corp. Question what benefits do you get with docker than without? Without I have Cgroups, my software is versioned i.e.: programm-X.X.X.jar / programm-X.X.X.rpm / whatever. Why do you get better / higher resource usage I mean you use an additional layer you could do the same with just CGroups. How do you do service discovery? Etcd could be installed without docker?! How do you auto-scaling without a cloud? I mean even without docker auto-scaling is trivial.
tomcam 11y ago
Why are shell scripts yuckworthy?
- mdaniel 11y ago
  I can't speak for the commenter, but for me the ... concern ... with using shell scripts is the complete lack of understanding of what they are going to do. It's not exactly a lack of trust for the Kubernetes team as much as when it goes toes-up (not if), then one needs to be able to understand what it did in order to understand what it didn't. I haven't used their cluster provisioning in a while, but back when I did it made a lot of modifications to AWS before falling over, at which point I need to either undo those changes or the scripts need to be smart enough to resume where they left off. Shell scripts are not well suited to that purpose, in my experience. Yes, I am aware of their salt stack procedure, but it doesn't hold a candle to the simplicity of Kubernetes on CoreOS, which unlike the 23+ directories worth of salt things to read is something you would probably fit on 3 A4 sheets of paper: https://github.com/coreos/coreos-kubernetes/tree/master/multi-node/vagrant https://github.com/coreos/coreos-kubernetes/tree/master/mult... I provisioned a Vagrant cluster using the Kubernetes salt mechanism just for this comment, and the "Cockpit" username and password didn't work after it was finished. Which salt config contains that information? Beats me. I'm thankful that `vagrant ssh master` did as expected. I recognize that the previous paragraph is just _my_ experience and _my_ preferences, but the anecdote is the reason why (in my experience) one must understand what the "magic" deployment process is doing, and only after that voluntarily cede control to the scripts merely as a labor-saving tactic. It does me no good to have a cluster brought to life that I then have zero idea how to maintain.

by gtaylor 11y ago

merb 11y ago

wow so instead of running my configuration management stuff I know need to either: - be on aws - be on gcloud - any other cloud that has docker support out of the box OR: I need docker + a configuration management to setup my environment, or I need to somehow manage my coreos configs. Oh and finally don't forget to run a network over a network cause it's dockerish so for most clouds this means we run a network on a network which runs a network. docker adds so much complexity. people just don't see this right upfront and use most of their time into these stuff, but there are easier ways to deploy. YES if you are really big and if your servers needs to scale way beyond the most than you need it probably since configuration management won't help you and setting up servers even in the cloud take some time looking at some netflix articles. however i just don't get it why people use their time for docker when there are other things to do in their programs.

pbecotte 11y ago
But it doesn't really, it is just another tool to learn like your configuration management software. Honestly, learning Docker to a high level was WAY easier than getting my Chef knowledge to a mediocre level. Further, it has other benefits like being MUCH easier to set up well isolated dev environments with Docker than with Chef. Since we switched we can usually have someone running the dev environment and tests within an hour or so of unboxing their laptop. With chef running the whole thing from scratch took that long by itself (after you installed all the software) and usually failed. And the idea that it is too complicated for something simple- I built a sample blogging app that you can deploy to a digital ocean or ec2 machine with like 3 or 4 commands (https://github.com/pbecotte/devblog https://github.com/pbecotte/devblog). The system we have built at work has 8 separate services between a bunch of data stores, background workers, and a couple different pieces of our app- but Docker allows you to run the entire environment on your local machine and then deploy that same setup to our cluster, without any differences to account for making all those apps run on one VM for your local environment (which can get bad when one service requires a different version of a ruby gem than another for example).
- fpoling 11y ago
  I second that. Docker makes it very easy to separate the whole application into a bunch of isolated services and then reproduce the whole production environment on a laptop for a very convenient development.
- merb 11y ago
  How do you do the same thing without a cloud Environment? Do you install the software manually? Do you configure your network manually? Do you configure your os manually? Do you install the docker daemon manually? What about installations behind firewalls? Or about code that shouldn't belong to the docker registry since it shouldn't be pushed over the internet ? etc... Why do you have 8 separted services anyway? How many people does your company have? for 8 services you should at least have 8 * 3 people.
gtaylor 11y ago
> wow so instead of running my configuration management stuff I know need to either: - be on aws - be on gcloud - any other cloud that has docker support out of the box No. You need a networked computer with a quasi-recent kernel. There are no requirements to use a cloud provider or to use AWS/GCP. > OR: I need docker + a configuration management to setup my environment, or I need to somehow manage my coreos configs. Oh and finally don't forget to run a network over a network cause it's dockerish so for most clouds this means we run a network on a network which runs a network. This sounds like aimless rambling. You don't have to use any of this. You can, of course, but I can do some nasty stuff without Docker as well. Like I mentioned with Container Engine, setting up a Kubernetes cluster is like two mouse clicks. You can get as simple as ECS, or you can roll your own from the ground up. You have the option to pick a point on a spectrum. With ECS, it's Amazon's way or the highway. > however i just don't get it why people use their time for docker when there are other things to do in their programs. Because it saves us loads of time developing, testing, and deploying our systems. There is more initial setup work, but after that we are deploying images seamlessly, have a great rolling upgrade and rollback story out of the box, and get a lot of other bonuses like service discovery, auto-scaling (vertically and horizontally), and much better (higher) resource usage levels. And we're far from a mega-corp. To me, it sounds like you may have skimmed some, perhaps even played a bit. But you ran into a snag, threw up your hands, and have summarily dismissed an entire ecosystem after your experience(s). I see enough un-informed or 100% incorrect things above to think you're missing some details.
- merb 11y ago
  > Because it saves us loads of time developing, testing, and deploying our systems. There is more initial setup work, but after that we are deploying images seamlessly, have a great rolling upgrade and rollback story out of the box, and get a lot of other bonuses like service discovery, auto-scaling (vertically and horizontally), and much better (higher) resource usage levels. And we're far from a mega-corp. Question what benefits do you get with docker than without? Without I have Cgroups, my software is versioned i.e.: programm-X.X.X.jar / programm-X.X.X.rpm / whatever. Why do you get better / higher resource usage I mean you use an additional layer you could do the same with just CGroups. How do you do service discovery? Etcd could be installed without docker?! How do you auto-scaling without a cloud? I mean even without docker auto-scaling is trivial.

tomcam 11y ago

Why are shell scripts yuckworthy?

mdaniel 11y ago
I can't speak for the commenter, but for me the ... concern ... with using shell scripts is the complete lack of understanding of what they are going to do. It's not exactly a lack of trust for the Kubernetes team as much as when it goes toes-up (not if), then one needs to be able to understand what it did in order to understand what it didn't. I haven't used their cluster provisioning in a while, but back when I did it made a lot of modifications to AWS before falling over, at which point I need to either undo those changes or the scripts need to be smart enough to resume where they left off. Shell scripts are not well suited to that purpose, in my experience. Yes, I am aware of their salt stack procedure, but it doesn't hold a candle to the simplicity of Kubernetes on CoreOS, which unlike the 23+ directories worth of salt things to read is something you would probably fit on 3 A4 sheets of paper: https://github.com/coreos/coreos-kubernetes/tree/master/multi-node/vagrant https://github.com/coreos/coreos-kubernetes/tree/master/mult... I provisioned a Vagrant cluster using the Kubernetes salt mechanism just for this comment, and the "Cockpit" username and password didn't work after it was finished. Which salt config contains that information? Beats me. I'm thankful that `vagrant ssh master` did as expected. I recognize that the previous paragraph is just _my_ experience and _my_ preferences, but the anecdote is the reason why (in my experience) one must understand what the "magic" deployment process is doing, and only after that voluntarily cede control to the scripts merely as a labor-saving tactic. It does me no good to have a cluster brought to life that I then have zero idea how to maintain.