6 ms·
AWS Copilot
- twillin 6y agoThis really seems like it could be a future replacement for Elastic Beanstalk. Especially when they finalize the ability to provision storage infrastructure.
- polysaturate 6y agoThat’d be nice. It was a good concept but I really didn’t like how it handled some of the application commands with yaml files.
- judge2020 6y agoHopefully. Elastic Beanstalk is a product that "works" and is "maintained", but when it fails it crashes hard. At least for the PHP environment, updating the environment variable configuration can take 2+ minutes, and non-immutable app deployments with 2 hosts can take over 10 minutes to pass health checks. And finally, if it fails to do something (eg. a deploy command fails on the instance), it will be stuck in the "error" state for 5-15 minutes where you can't do anything to the environment and must wait for it to clear to fix your bad deploy.
- judge2020 6y agoI should clarify that the app deploy itself can take under 5 minutes if non-immutable; immutable deployments are the ones that can take 15 minutes or more.
- freeqaz 6y agoAmen I thought I was just doing something wrong! What is the best alternative to Elastic Beanstalk on AWS? Is Kubernetes overkill?
- kohidavez 6y agoHi!! I’m a developer on AWS Copilot! We’d love your feedback! Homebrew: brew install aws/tap/copilot-cli Docs: https://aws.github.io/copilot-cli/ https://aws.github.io/copilot-cli/ Feedback & Requests: https://github.com/aws/copilot-cli/issues/new https://github.com/aws/copilot-cli/issues/new
- anderiv 6y agoThanks for your work on this! Wondering if it’s possible to use this with our own self-managed ASG of ECS Host instances. I trolled through the GH repo quick, but couldn’t find any definitive answer.
- kohidavez 6y agoNot yet - we’ve started out with Fargate but we have a roadmap item to support EC2 and resource imports: https://github.com/aws/copilot-cli/issues/1094 https://github.com/aws/copilot-cli/issues/1094
- deleted 6y ago[deleted]
- emmelaich 6y agoIt's a pity you used "copilot", because that name is already used for Red Hat's admin product.
- gavinray 6y agoI happened to catch this on the Github daily trending list a week or so back, really cool. For similar tools (though this one is ECS + Fargate), also see: https://somanymachines.com/fargate https://somanymachines.com/fargate https://github.com/awslabs/fargatecli https://github.com/awslabs/fargatecli I'm pretty particular to Pulumi, and defining infra in typed programming languages with full IDE tooling. AWS CDK is the AWS-centric equivalent, and that's a great and under-represented tool as well IMO. https://www.pulumi.com/containers/ https://www.pulumi.com/containers/ https://docs.aws.amazon.com/cdk/latest/guide/home.html https://docs.aws.amazon.com/cdk/latest/guide/home.html My issue with ECS/Fargate is that on the surface, it seems amazing. And so you deploy your container, and that's relatively easy. But now you have just a container on HTTP, on a bare IP. You probably want HTTPS and a domain name attached. So then you learn that you need to: - Create a Route53 hosted zone for the domain - Use AWS Cert Manager to provision certificates for the domain - Create a load balancer, and figure out how to attach it to the ECS/Fargate service - Wire up the LB address to the record set and configure the termination for HTTPS And to do something like change an environment variable, you have to create an entirely new "task definition", deploy the task revision, and then spin up the new service and stop the old one. This takes about 5 not-particularly-easy to find or navigate screens in the AWS console. This is what it ended up looking like, for me to deploy a Fargate service: https://gist.github.com/GavinRay97/fbedfb18fe852bf625451a4175695ad4#file-hasura-fargate-https-alb-route53-cert-pulumi-ts https://gist.github.com/GavinRay97/fbedfb18fe852bf625451a417... It's great that these tools can help reduce the burden of doing this but man is it a painful experience if you're not a wizard at DevOps. Google Cloud Run is 1-2 bash commands to deploy a containerized app on HTTPS.
- kohidavez 6y agoCopilot can also help set up HTTPS/ACM Certs/Subdomains: https://github.com/aws/copilot-cli/wiki/Applications#additional-app-configurations https://github.com/aws/copilot-cli/wiki/Applications#additio... I love the cloudrun style service too - but one thing I think is cool about running on ECS/Fargate is that once you outgrow the constraints of CloudRun style services - you can drop down to the more granular infrastructure to make tweaks and adjustments. Just my 2c :)
- throw03172019 6y agoThis reminds me of Convox. Very simple to get started and runs on ECS. Copilot has less features as of now but looks like it’s going in a good direction. (Happy convox customer)
- miraj18 6y agoThis is cool
- vazamb 6y agoWhat are the advantages of this over CDK? It's trivially easy to spin up a loadbalanced service https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ecs-patterns.ApplicationLoadBalancedFargateService.html https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws...
- fulafel 6y agoCDK is terrible on so many levels, it's possible that this is less so.
- fulafel 6y agoExpanding failures of CDK due to popular demand: - It introduces an edit-test-debug cycle from hell, it doesn't do what you think from the mental model of a user-friendly CF template generator library (doesn't statically ensure stuff it emits actually works even superficially) - It builds upon an already difficult and leaky abstraction (CloudFormation) so you just get additional failure modes and cognitive burden - Glacially slow iteration, can easily take 10-30 mins to teardown/setup your app due to CloudFormation. - fails to provide upsides from the costly tradeoff it makes to jump into turing complete programming languages, like 3rd party cdk components or iterating expressions at a repl - don't even get me started on the npm dependency hell (you'll randomly be debugging mystic npm warnings already on the next day when installing a new submodule of cdk deps, when version mismatches appear), the other-languages-to-npm bridges, the "you can't use this without an IntelliSense style IDE" API design, buggy cli tools, etc To summarize it doesn't manage complexity or give you a good abstraction. Probably because it tries to take on all of AWS functionality at once. (Also the GP linked Fargate example is far from the functionality from the copilot example, like enabling you to actually dev and deploy the app - ecr, codebuild, codepipeline etc)
- kohidavez 6y agoHiya! My team is also the team that writes and maintains the CDK construct you linked to. I think it’s different approach - that’s all. Copilot Is more focused around a full developer experience including setting up CI/CD, deployment environments and operational tooling like logs, stats, etc.
- jaxxstorm 6y agoI wrote a blog post about this that did the rounds last year about the topic I believe Copilot is trying to solve: https://leebriggs.co.uk/blog/2019/04/13/the-fargate-illusion.html https://leebriggs.co.uk/blog/2019/04/13/the-fargate-illusion... Summary: the marketing spiel about how easy fargate is to use and get up and running isn't actually how the experience is. I'm hoping Copilot improves that experience.
- kohidavez 6y agoPlease give it a shot and let us know!
- dunk010 6y agoReally great article. I've walked the same path and thought the same thoughts, and always felt somehow that it was just me. Here's hoping that Copilot lowers the barrier for all developers - especially ones who just have a small site or hobby project.
- deleted 6y ago[deleted]
- noisy_boy 6y agoNot directly related to copilot but in context of ever increasing yml/json configurations for cloud apps, am I the only one who feels overwhelmed with the configuration options? I see tons of articles with just drop this json here and that yml there and run this cli command - where is the reference for these json/ymls? Does everyone fully know/understands the structure/options? How do people keep track of the changes to that between upgrades? I feel increasingly dumb the more I see these configs.
- Mandatum 6y agoThere's definitely room for config-viewer plugins in VSCode, Sublime, etc. to help write and maintain the 100's of config files we have to be knowledgeable about in DevOps these days. Does anyone have good suggestions in this space? Or do I just have to google docs and reference documentation when I need to figure out what the hell all these parameters mean?
- f6v 6y agoI think clouds have been trying to fool developers into thinking that cloud is easy. Don't feel bad, building a reliable infrastructure is incredibly hard. There're ton of services, some of them are redundant, and it takes a lot of time to discover all the nitty-gritty.
- jmchuster 6y agoMaybe it's messaging intended to be for grizzled old veterans going "look how we've taken the old complicated thing and given you powerful tools to allow you to manage it more easily with less steps" but then gets interpreted by neophytes as "look how easy it is for a complete beginner to come in and set up a complicated system".
- gk1 6y agoGreat observation. A lot of messaging for cloud products is comparing against legacy or dominant solutions. A beginner doesn’t have that reference point.
- vadiml 6y agoThe example yaml file contains cpu: 1024, but the text says deploy on 1 cpu?
- deleted 6y ago[deleted]
- omni 6y agoIn ECS land, a single vCPU is described as 1024 "CPU units" that can then be distributed more granularly. This would allow you, for example, to evenly distribute a single CPU across 4 processes.
- ookblah 6y agohaving tried everything from convox to fargatecli, deployfish ,etc. in the end i just ended up using pulumi to set up the basic infrastructure components and writing my own bash scripts to deploy tasks and services to ECS using the AWS cli. there was way too much magic going on and i wasn't confident i could easily roll back or do modifications if the CLI didn't like something in particular. deployments based on terraform or cloudformation were particular susceptible to this.
- justincormack 6y agoNote also that the Docker integration into Fargate was released at the same time, see https://aws.amazon.com/blogs/containers/aws-docker-collaborate-simplify-developer-experience/ https://aws.amazon.com/blogs/containers/aws-docker-collabora... and https://www.docker.com/blog/from-docker-straight-to-aws/ https://www.docker.com/blog/from-docker-straight-to-aws/ - this provides a simpler way of using Fargate, especially if you are already using Docker Compose. You can output cloudformation from this as well, but you don't have to understand the internals of ECS to use it. This is in beta and would love more feedback (here https://github.com/docker/ecs-plugin); https://github.com/docker/ecs-plugin); I worked on this at Docker with Amazon.
- gavinray 6y agoMy goodness, Docker Compose file to production stack?! My dreams have come true. The arguments for k8s I have is that the local and production setups are identical. It's a PITA to run k8s locally, but the alternative is that you use Docker Compose local, and then deploy to k8s, which means two sets of configs. And that's terrible. Or, you use Docker Compose locally, and fragment all of the containers into individual ECS/Fargate services which you write the deploy pipeline for individually. These were your best options, for one glaring reason: You can't use a docker-compose.yaml in production. (I know, I know, Kompose, but it doesn't always work quite right and you now have a k8s cluster). Really excited to try this out, thanks for the work on it!
- drej 6y agoI probably missed it in the blogpost and github readme, but is there a good way to "export" this into a CloudFormation/Terraform template? After all, this seems to spawn a few things (networking, mostly), so I want to make sure I can automate any further deployments. Thanks!
- jeremya 6y agoCopilot generates CloudFormation templates based on the Copilot-specific config files. There’s a place where you can add your own additional resources in CloudFormation syntax IIRC. It also has the ability to create a CodePipeline for app deployment. That creation is also done by generating a CloudFormation template and deploying it for the user. I’m not sure why it’s not clearly documented that this is happening, but it is a selling point to me because if I want to, I can “eject” from Copilot and just maintain/enhance the CloudFormation templates.
- Aeolun 6y agoI have a hard time believing anyone would consider ejecting to CloudFormation a selling point.
- llarsson 6y agoCould be nice to version control them. Some CLI stuff looks cool on a blog, but do you want to remember what commands you typed, and in what order, to bring up what production looked like two weeks ago manually?
- dkarl 6y agoIt's a cautious way to start working with a new tool. It's AWS, but there will still be bugs, especially for people with large or unusual deployments. You could use the tool to generate CloudFormation configs and then put them through a PR process. After six months of this with no bogus configs generated by the tool, you might decide you're on a well-paved road and make it part of your CI/CD pipeline.
- kohidavez 6y ago
- wballard 6y agoThis is pretty cool - very close to https://shipula.userbugreport.com/ https://shipula.userbugreport.com/ that I've been cooking up to make my own life easier.