11 ms·
AWS Fargate Deep Dive
- codewithcheese 7y agoI would love a managed Kubernetes Deployment/Job/StatefulSet. Forget managing the cluster or the node, just allow me to "apply" a Deployment config with associated Service straight to the cloud. I will tell you my resource limits bill me accordingly. I hope Google Cloud or AWS is working on that. That would have a much wider impact then Fargate.
- ricardbejarano 7y agoAnd have unlimited namespaces, letting you choose the same as another deployment you have or a new one. Make it have a default deny all NetworkPolicy and you are all set. I'd love to see this too.
- wikibob 7y agoCheck out Google’s CloudRun https://cloud.google.com/run/ https://cloud.google.com/run/
- base 7y agoThere is the old AWS Beanstalk https://aws.amazon.com/elasticbeanstalk/ https://aws.amazon.com/elasticbeanstalk/
- fovc 7y agoFor people who worry about security (either sincere or tick the box types): what are the pros and cons of managed containers? It seems like you get a reduction in attack surface but also have fewer tools at your disposal
- amsully 7y agoDon't know from a Fargate perspective but Elastic Container Service (ECS) deploys EC2 servers that do not pass the CIS Benchmark. I don't believe you gain much from a security perspective.
- x3n0ph3n3 7y agoYou can roll out your own EC2 instances and Auto Scaling Group for ECS and control the security on them yourself.
- OhSoHumble 7y agoThis is incredibly true. The only requirement for an ECS cluster member is to be running the ECS agent - which is a Golang binary. You're free to run a CIS hardened image if you desire to do so.
- ed6612 7y agoThis is how we roll. CIS as base, Packer to customize (ecs agent, docker) into own AMIs.
- fovc 7y agoAre there OSS or commercial AMIs that have been hardened? Maybe some RHEL or CentOS?
- OhSoHumble 7y agoYeah, if you look at the AWS image marketplace then you'll find some.
- zo1 7y agoFor some odd reason, I find the AWS marketplace a bit suspect looking. Not saying that it is, just that that's my impression of it.
- bbgm 7y agoOne of the key things to remember about Fargate is the following. Each Fargate task has its own isolation boundary and does not share the underlying kernel, CPU resources, memory resources, or elastic network interface with another task. (Source: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...) The other part is patching. We are (I work at AWS) responsible for patching the underlying hosts. More details at https://docs.aws.amazon.com/AmazonECS/latest/developerguide/platform_versions.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...
- nnx 7y ago> Each Fargate task has its own isolation boundary and does not share the underlying kernel, CPU resources, memory resources, or elastic network interface with another task. Isn't Lambda the same as they're both using Firecracker under the hood isn't it?
- bbgm 7y agoWith or without Firecracker the isolation models are similar.
- dodobirdlord 7y agoYes
- staticassertion 7y agoFargate solves problems like patch management, but there are other issues. If an attacker were to compromise a Fargate service they'd have less to work with, but otherwise there are a lot of advantages they'd have over EC2. On EC2 I can deploy monitoring tools like OSQuery, or rely on audit/ other OS subsystems to monitor for suspicious behavior. On Fargate I don't think anything like this is possible.
- tptacek 7y agoWhat security problem would you be addressing with osquery that Fargate doesn't already address? If you're worried that AWS is owned up, it doesn't matter which AWS service you use.
- fovc 7y agoI was thinking of this very limitation when posting the original Q. If e.g., there a RCE issue with my PHP server on Fargate, as the attacker I have a foothold and there's no monitoring inside the container as I try to move laterally. I guess the same is true with EC2 backed containers though
- tptacek 7y agoYou can instrument the container itself (the environment in which RCE against PHP would provide an attacker). You can't instrument the container engine or the host server, because AWS owns the security of those. But AWS will do a better job with those than you will, or at least, your whole usage of AWS is premised on that.
- tbrock 7y agoI love AWS, I really do and I thought about using fargate because the promise of not managing your “cattle-like” servers is wonderful but they need to get the pricing within this stratosphere for it to not be a complete joke. I actually really like ECS and aware of how much time it would save me (a lot) and how much terraform I could delete (a ton) and it’s still not even close to worth it. Amazon usually nails this sort of thing, surprising that despite the operational value it provides nobody seems to be using it.
- roadofbones 7y agoI wish my company could use it, we run a bunch of standard up ECS clusters, and it'd be nice to not have to provision and maintain servers. Fargate wasn't worth it due to the price difference, and just how rigidly they enforce cpu and memory quantization.
- helpPeople 7y agoAmazon has become the vilian with pricing.
- solidasparagus 7y agoPricing is fine for small things (most people's side project, etc). Fargate still feels very young and I don't think it handles very high scale or resource intensive work all that well/affordably. I think that will come in time. From a cloud provider PoV, Fargate is a very hard problem - like Lambda except harder because the container might need to run forever.
- airocker 7y agoI believe vendor lock-in and lack of incremental value over EKS are the reasons. The amount of flexibility we lose by using Fargate is not compensated well enough by the ease of use offered.
- staticassertion 7y agoWhat lock-in? I'm genuinely curious, I can't imagine it would be hard at all to move my service from Fargate to anything else. It's just docker containers and DNS, I haven't done anything Fargate specific.
- mharroun 7y agoWe spend ~900$ a month on fargate to run our of our dev, stage, qa, and prod environments as well as some other services and sqs consumers. After the recent price decrease we looked at how much reserve instance would save us and the few hundred in savings would not make sense vs the over provisioning and need to dedicate resources to scaling and new tools to monitor individual containers. Note: do have some stuff in lambda but its package size restrictions limit us.
- philliphaydon 7y agoYour packages exceed 250mb??? Wow
- etaioinshrdlu 7y agoMy docker containers average about 5gb total. And I have lots of them. It is incredibly easy to become bloated. It is hard to stay bloat free
- philliphaydon 7y agoWe're talking about Lambda tho, executing a function, how much code is required to execute a function... The largest .net lambda I've managed to create is ~40mb which included chromium.
- dimgl 7y agoYou're getting downvoted but clearly there is something wrong with the way these devs are using lambdas.
- jon-wood 7y agoWhile Lambda is sold as “functions as a service” I have on several occasions made the particular function being served the router for a Ruby web application - with the Ruby runtime there’s really nothing stopping you running a complete Rails app in Lambda other than size limitations once you add a few too many gems.
- squid3 7y agoNodeChef is a good alternative where you don't have to do the tedious job of managing servers. https://www.nodechef.com/ https://www.nodechef.com/
- jjeaff 7y agoLooks nice. But I put in 2 apps with 4 cpu and 1gb of RAM each. And I selected MySQL database with 1gb of RAM and 30gb storage. And it quoted me $400/month.
- gravypod 7y agoAre they allowed to host a MongoDB for you? Isn't there something in the license about that not being allowed?
- detaro 7y agoNot of the versions they offer.
- haolez 7y agoI’m using Fargate for services that are CPU intensive (i.e. 24/7) and not reactive by nature. It’s been a good experience so far.
- zmmmmm 7y agoFor my poor brain still trying to cope with the enslaught of the huge number of all these cloud service features ... this sounds a lot like kubernetes ... is this just a proprietary version of that? Can someone differentiate them for me?
- srparish 7y agoMy understanding is with EKS or similar you still have to manually size your kubernetes cluster, that is you have to make sure there's enough hardware instances in your kubernetes cluster for whatever scaling you'll need. With fargate you're effectively using AWS's own cluster. Your service can scale up and down, and you only pay for the resources that your service actually uses.
- joseph 7y agoIt's a fully managed version of Amazon ECS (elastic container service). With Fargate, you don't need to manage the EC2 instances that make up the cluster, as is required with ECS. Early on, Amazon tried to avoid offering a managed Kubernetes service, and so they rolled their own container service in the form of ECS. Later they caved in and created EKS, their Kubernetes platform. ECS is still used as the underpinnings of some of their other services, such as Batch and Fargate.
- lkrubner 7y agoIt's not 100% fully managed. You still need to set auto-scaling rules. I find that mildly annoying.
- gingerlime 7y agoWhen I played with Fargate about a year ago or, this was its Achilles heel. I was hoping for a solution that would auto-scale very quickly, but the healthcheck intervals and minimum counts to consider a container "running" couldn't go below 30 seconds or so. As far as I recall, this is all configured on the load balancer, rather than directly inside Fargate. Somehow makes it feel less like a "fully managed" solution, but rather something you have to still tinker quite a bit with. (compared to Lambda, which you really don't have to worry about scaling at all) EDIT: [0] indicates that the minimum you can set is 10 seconds (minimum 2 intervals of 5 seconds to consider it "healthy"), if I understand it correctly [0] https://docs.aws.amazon.com/elasticloadbalancing/latest/application/target-group-health-checks.html https://docs.aws.amazon.com/elasticloadbalancing/latest/appl...
- peterwwillis 7y agoIn general, you should just start with whatever service AWS has that integrates the most features, and once you know what your technical requirements/limitations are, you'll know if you need to back up to a less integrated solution. Worst case, you're paying too much for a solution for a short time, but you have a working MVP.
- crucialfelix 7y agoI still haven't managed to SSH into a container (for Django). The best way I guess is SSM (systems security manager) which at least gives a web based console. codepipeline integration was time consuming to set up. You have to get it to create a json file with the image id and uh I'd have to consult my notes. All told, it was more complicated to set up than I expected.
- jon-wood 7y agoWe’re working on this at my workplace as a pre-requisite to a widespread deployment of Fargate, its the only blocker on moving out of Heroku for a large distributed system. The approach I’m going with is to have an EC2 host attached to the ECS cluster which people can schedule interactive tasks on. Coupled with some scripting (maybe a Lambda function if I decide to get fancy) we can then start a task for any given service on the instance with the same environment and IAM role, but with a command like /bin/yes just to keep it alive. Once that’s running users can SSH to the host instance and docker exec into the container for whatever command they actually wanted to run. It’s quite a bit more involved than Heroku’s run command, but initial prototypes seem to indicate it’ll work once we wrap it in some tooling.
- samvher 7y agoI like using Fargate for one-shot tasks that are easy to split up. I used it a couple of times for summary tasks on large batches of satellite data (100s of GBs). Set up a docker image that takes the month for which to do the analysis as environment variable and then launch 50 or so Fargate tasks in parallel. Fairly easy to set up and can save quite a lot of time. If it's for short running jobs the increased price is not much of an issue. For more complicated, long-running services I feel like I would prefer managed Kubernetes.
- fulafel 7y agoCan I easily SSH into or otherwise interactively get a shell into a Fargate container? I think this is a minimum debuggability requirment for these kinds of services. https://github.com/aws/containers-roadmap/issues/187 https://github.com/aws/containers-roadmap/issues/187 sounds like the answer is "no"?
- flurie 7y agoThe short answer is no. The long answer is that you could theoretically explicitly set up a user for auth, run sshd in fargate containers, and shell in, but it’s not going to be worth it as anything other than a toy example.
- fulafel 7y agoI wonder what's behind this functionality gap in the managed services. After all we take docker exec, podman exec, kubectl exec etc for granted in troubleshooting.
- foolfoolz 7y agoif you need to ssh in to a box you are not ready for fargate or lambda. think about what commands you would actually run on the shell. what are you looking for or reading? you can find all that information from cloudwatch or other logs / metric services. you'll find at scale, sshing into a particular ec2 box also doesnt make sense
- fulafel 7y agoSooner or later you end up needing to debug in production to narrow down phenomena that for some reason don't replicate. Repl du jour, aws cli, lsof, tcpdump, netcat, perf, netstat, iptables, poking around in /proc, valgrind, pdb, replacing binaries with shell scripts that log their args, rerunning various subsystems from the shell with debug/verbose switches, retrieving badly configured core dumps etc. If you've been spared from these kinds of issues, be thankful for your so far sheltered life :) If your problem is reproducible at will in prod (big if), theoretically you can bake each thing you try into your container build and debug its automation at each step, often slowing the debugging process down prohibitively.
- 013a 7y agoDoes anyone remember that time at Re:Invent 2017 when they announced Fargate, and said that Fargate was coming to EKS "soon"? Let's put odds on which is released first: Fargate for EKS, or Half Life 3.
- nickthemagicman 7y agoIs EKS a serious competitor to this? It seems like it would be and the bonus of no lockin. What's the advantage of Fargate over EKS?
- sebasmurphy 7y agoThis product always makes me think of Aqua Teen Hunger Force. I wonder if that's where the name originated. https://youtu.be/uOd7HQoKxcU?t=50 https://youtu.be/uOd7HQoKxcU?t=50