5 ms·
If you are on AWS, try ECS first, it is much simpler and has main features you need: HA, autoscaling, version control. When you'll mature from using Docker to
by auslander 8y ago
If you are on AWS, try ECS first, it is much simpler and has main features you need: HA, autoscaling, version control.
When you'll mature from using Docker to high volume production, you should ditch containers at all, they are good for prototyping and testing, but not for production loads and production security.
- wikibob 8y agoECS is simple to get started, but does not have a vast array of features that are necessary to run a service reliably at scale. It is possible to add many of these to ECS, but it’s left as an exercise to the reader to write up various API’s, cloudwatch, Lambda, etc. I can provide more specific details if there’s questions.
- auslander 8y agoThank you. Yes, could you name the features ECS is missing for running at scale reliably, please?
- shawabawa3 8y agoI've recently moved from ECS to Kubernetes, and much prefer kubernetes In general ECS is fine when you have 1 or 2 services, but once you have 10s or 100s it gets pretty unusable. Logging: While ECS ships everything to cloudwatch, actually accessing them is a nightmare and usually more effort than it's worth. It's very easy on k8s to get logs shipped from every container to elasticsearch and browse with kibana. Autoscaling: Possibly less of an issue now that fargate's a thing - but fargate's expensive. Kubernetes makes it easy to set up both node level autoscaling, and deployment level autoscaling Ingress: With ECS you have to set up loadbalancers for every service individually - in k8s once the controller's properly set up every service defines its own ingress, and the only manual change you have to do is create a DNS entry (only if you can't use a wildcard) Metrics: Prometheus is very ingrained in kubernetes, and you get a wealth of information from every service almost "for free" once you've installed it. And more: service meshes, secret control (through e.g. vault), declarative definitions, provider agnostic Kubernetes has a huge startup cost and learning curve, but is incredibly powerful once you're up and running
- auslander 8y agoAutoscaling: No need for Fargate. Create 'AWS::ApplicationAutoScaling::ScalableTarget' and 'AWS::ApplicationAutoScaling::ScalingPolicy' for your 'AWS::ECS::Service' resource. Logging: Subscribe the ElasticSearch Lambda to CloudWatch Logs group to have all logs in ElasticSearch domain. Kibana is included in 'AWS::Elasticsearch::Domain' resource. Ingress: Yes, one ALB per one autoscaled ECS Service, I don't see it as a problem. Metrics, And more: not sure I see the benefits. CloudWatch has miriad of metrics, you can get lost there :) It might look hard, but here is great starting point : https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-ecs.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui.... Yes, if you want to use AWS, use Cloudformation templates, there is no way around it :)
- baursak 8y agoI couldn't figure out how to run master-slave configuration on ECS.
- deleted 8y ago[deleted]
- ianseyer 8y agoThat's a rather bold claim, care to elaborate?
- auslander 8y agoYes, happy to. Which claim you would like to focus on?
- ianseyer 8y agoContainers not being good for production.
- auslander 8y agoExcellent point :) So. First of all, you'd not have to worry about following, if you don't use containers. If your app, running in Docker container, is compromised, like PHP webshell, it might try to escape the container. What capabilities you granted for your containers? CAP_SYS_ADMIN, CAP_NET_ADMIN or you have no idea? This is just one example of escaping https://www.twistlock.com/2017/12/27/escaping-docker-container-using-waitid-cve-2017-5123 https://www.twistlock.com/2017/12/27/escaping-docker-contain... and lets talk about namespaces, like user namespace. Is root inside container is a root user on host system? Does it feels like production ready system?
- thinkmassive 8y agoIf the same compromised app is running natively then the entire system is now compromised. Capabilities and namespaces can be used with or without containers.
- auslander 8y ago1. You may have different apps, owned by different owners, running as containers on same underlying host machine. 2. Not running docker means you can lock your httpd by chroot, FreeBSD jail or OpenBSD pledge.
- nickthemagicman 8y agoOrchestration proved to be difficult in ECS for me.
- otterley 8y agoCan you elaborate? Can you give an example of a specific problem you ran into with ECS, and how you were able to easily solve it with K8S?
- nickthemagicman 8y agoI couldn't get my nginx container to communicate with my app container in ECS however it all worked locally. And theres very little docs or troubleshooting resources.
- bdcravens 8y agoIf you're just launching a standard app, pretty easy. However, anything complex requires writing a ton of AWS SDK code. (For example, we launch a spot instance with a given task definition, assign a given customer's work to it, then shut it down when the work is finished, and then on to the next customer)
- auslander 8y agoToo complex, KISS principle rulez :) Creating ECS Cluster on spot instances seems like a way to go. Haven't tried myself yet, in to-do's :)
- bdcravens 8y agoWe had a simple setup before (and have used spot instances everywhere it made sense). Without getting into the specifics of the use case, the ability to target a specific instance for specific backend processing tasks became necessary. Rest assured, it's not "too complex", only "as complex as necessary". Be careful not to respond with a rote "you're doing it wrong"-ism when discussing technical merits.
- bdcravens 8y agoWe've had good luck with ECS (though occasionally I'll find it doesn't shut old tasks down). Honestly though I feel like ECS will go the way of services like SimpleDB, Cloud Search, etc. (Never going away, but a secondary offering)
- auslander 8y agoLaw of natural selection. Good thing is AWS would never depreciate an actively used service.
- threeseed 8y agoECS has a docker size limit of 4GB before it will crash silently with no user facing error message. It is also is pretty buggy with the agent sometimes not starting and paying for nodes that aren't used. Also containers are better for production security than just having apps sitting side by side on the same disk.