10 ms·
I spent the last few years developing software on AWS. I honestly cannot imagine not using it. Once you get good at infrastructure as code (CDK), microservices,
by fdgsdfogijq 5y ago
I spent the last few years developing software on AWS. I honestly cannot imagine not using it. Once you get good at infrastructure as code (CDK), microservices, and how everything wires together, any other technical stack is just painful. I still believe that engineers are completely underestimating productivity gains in people that are good at AWS. There is just so much less BS in terms of systems working, everything clicks together, you arent stitching a million open source libraries to build a stack. Billions of dollars of effort are behind making it as seamless as possible.
- elorant 5y agoWould you recommend any resources/book for someone who hasn't got a clue what all the services at AWS are about?
- fdgsdfogijq 5y agoI would start by understanding the infrastructure as code paradigm, built by CDK. With CDK, I would build some kind of webapp, incrementally adding more resources (lambdas, databases, etc). The true value is that you should never modify resources you provision in the console and only do it through code. Once you start developing software like this, it will be very hard to go back to the old way.
- lambdaba 5y agoWhere does this sit in relation with Terraform? Is Terraform not able to do everything this does? I was aware of something from AWS called CloudFormation templates or something but I heard that was too low-level and idiosyncratic, also JSON-based and not expressive enough. Sorry if this is a bit confused, just want some clarification, and thought that Terraform was the gold standard.
- pojzon 5y agoTerraform is using a delarative DSL. CloudFormation is similar in that regard. Using a declarative language is both a pro and con. Pros of that solution is that you are restricted in what you can do, which often results in simplier and cleaner code. Cons are that again you are restricted in what you can do. Now about CDK is just your preferred language library to write infrastructure in imperative way. For example you write java code that deploys infrastructure. When using a real programming language you have ofcourse a lot more freedom in what you do, but ofcourse that comes at the cost of complexity. If your developers are “creative” it can be very hard to understand wth is happening in code. —- So yes many ppl consider Terraform as a golden standard (me included) but after many years of working with it, like anything in IT it has own flaws and you may want to consider other solutions for specific projects.
- jen20 5y agoCDK is also declarative - it generates a CloudFormation YAML file, so cannot possibly be anything else. Declarative is a property of a programming model, not a language.
- pojzon 5y agoInbetween using CDK and generating CF Template you can be very “creative”, thats where the imperative part lies.
- 8note 5y agoI think there's a much stronger argument that cloudformation deploying your stacks gets to being imperative than the way you produce your cloudformation file. The file is still declarative regardless, and the functions you write to create that file don't per se have side effects outside that file
- Too 5y agoIt’s possible, and very easy, to generate yaml with a bunch of for-loops and side-effects in an imperative language. So without knowing anything about cdk, even if it is declarative, I don’t think your conclusion is applicable. Terraform, the language, works differently. While it has variables, you can not store state in variables and depend on side effects based on the order you read/write such variables. Every assignment and execution order of resources is purely based on dependency order and not based on the order of the lines in your code.
- jen20 5y ago- CloudFormation is the native offering from AWS. Templates are submitted in the form of YAML or JSON documents. - CDK compiles Typescript and other languages down to CloudFormation Templates - Terraform uses the AWS API directly via an AWS-specific provider, and does not typically use CloudFormation stacks (though this can be useful, in some circumstances!) - Pulumi uses TypeScript and other languages to drive an engine similar to Terraform (and the Terraform AWS provider is one of the options for provisioning AWS resources). Hopefully this clears it up?
- deleted 5y ago[deleted]
- holler 5y agoI'd recommend learning raw cloudformation first since it's the primitive cdk compiles to & being able to read/understand it will give someone full confidence of their stack.
- fdgsdfogijq 5y agoThis is how I learned. Its a great point, might be tedious to do at this point though
- sethhochberg 5y agoThere's so much going on within AWS I'd suggest approaching it the other way and identifying a few examples of things you'd need, and then researching how AWS implements those things - if you need a mainstream relational database, AWS has RDS or Aurora, etc. Need to make sure your RDS database is only accessible from your private backend app? Thats a Security Group. Need somewhere to run that app as a Docker image? ECS or EKS... which may require learning a bit about IAM so you can make sure your own user / deployment user has the right permissions to manage all these resources. Its pretty straightforward to pull a thread - but there are lots of threads you could pull. But this has been shared here before and is still pretty good: https://adayinthelifeof.nl/2020/05/20/aws.html https://adayinthelifeof.nl/2020/05/20/aws.html
- jayair 5y agoIf you are looking for a guide on building full-stack apps on AWS, check out: https://serverless-stack.com/guide.html https://serverless-stack.com/guide.html It covers a small subset of AWS services but it is very comprehensive in what it covers.
- treesknees 5y agoThe author touched on this, you can use cloud services without the complexity and billing nightmare that is AWS. It's not AWS vs DIY. >Is it actually good as default choice to host a software system of any scale? Let us go through some arguments against going all-in with AWS and why many companies and individual side-project developers might be better off using something else. To be clear, the author is suggesting that AWS doesn't need to be your first choice in smaller setups and side projects. I don't think they're intending to say people shouldn't use AWS under any circumstance. There are plenty of smaller cloud service providers that can offer similar functionality to AWS that probably make more sense for smaller-scale uses. I personally love Linode, and you can definitely pull together their various services to build something without doing it from the ground up on VMs.
- fdgsdfogijq 5y agoMaybe, but I'll just say again, after a few years of day in and out AWS development, I can build a full blown website, with queues, microservices, RDS, react app, monitoring, alarms, logs, authentication, in a few weeks. Everything is rock solid, done by the book, and CHEAP. I think you are underestimating the power of this
- jjeaff 5y agoI can deploy most all of those services to my k3s cluster just as quickly using their respective helm charts.
- zmmmmm 5y agoI guess this sounds dismissive, but it just doesn't sound that impressive. Anybody can do that on any cloud provider in a mostly vendor agnostic way with some docker-compose or kubernetes yamls / terraform scripts. I think you've lost touch with where the non-AWS world is up to and are assuming it's still where it was in 2012. It's not.
- IceWreck 5y agoInfrastructure as code can work on any cloud provider including smaller ones like Digital Ocean, Linode, OVH, Hetzner, etc which are mentioned in the article. A lot of the things you said apply just as much to them as to AWS. In my opinion using AWS is fine, as long as you use cloud agnostic stuff and avoid AWS specific services. That way if you ever have to switch, you just have to rewrite the provisioning part of your infra-as-code, everything else can be easily migrated over with some config changes.
- oneplane 5y agoIf you are not using cloud native features you are essentially just using a virtual datacenter at which point AWS is an expensive choice. On the other hand, if you use Kinesis, Lambdas, RDS, VPC peering, and IAM accross all of them including EC2 and EKS, then no, none of those other options can even come close. Purely the lack of system-wide IAM alone would be a dealbreaker. This is also why being 'on the cloud' and 'cloud native' is very different. While the latter is a bit too buzzwordy for my taste, it does show the difference between "playing datacenter" and "getting stuff done in new ways, both cheaper and faster". Do not use AWS if you just run a few virtual machines and some basic networking. That also applies to GCP and Azure. It's too expensive for what you need, and too easy to misconfigure and shoot yourself in the foot. On the other hand, if you need API-driven infrastructure with a large gradient between IaaS and SaaS to pick from, then by all means, use AWS.
- zarzavat 5y ago> If you are not using cloud native features you are essentially just using a virtual datacenter at which point AWS is an expensive choice. Hence “you probably don’t need AWS”. > On the other hand, if you use Kinesis, Lambdas, RDS, VPC peering, and IAM accross all of them including EC2 and EKS, then no, none of those other options can even come close. Purely the lack of system-wide IAM alone would be a dealbreaker. Hence you are locked in and have zero negotiating position. Your business has an existential risk based on AWS - assuming your code is critical to the business and could not practically be rewritten because of all this special sauce you’re using. That might be okay but I sure hope your CTO made that decision consciously.
- pojzon 5y agoDeployed few services on ecs, nat gateway, shared elb, certificate and route53. Total cost 30$ per day ~900$ per month. You can accomplish the same for a fraction of that cost. The convenience you talk about comes with a huge bill. If I add reserved instances/savings plan, probably gonna drop to 600-700$, but still… I can setup that myself on my local lab for around 5$ per month. So the knowledge required to setup that and hardware is worth around 695$ per month. Thats why Cloud Providers are biggest employers of old sysadmins ;)
- fdgsdfogijq 5y agoThis is true, but I will argue again that I am proficient enough with AWS to deploy a full stack with 10+ lambdas, DNS, certs, RDS, step functions, etc for < 5 dollars a month. You need to understand what to use and what not to. Most of that bill is from the nat gateway/private VPC, which you really dont need
- freedomben 5y agoRDS for < 5 dollars a month?
- mitjam 5y agoHow? Last time I looked the smallest RDS for MySQL came at 11 USD.
- douglasisshiny 5y ago>Deployed few services on ecs, nat gateway, shared elb, certificate and route53. >Total cost 30$ per day ~900$ per month. Ummm... I think you're doing something incorrectly. The biggest line items on that list are ELB (assuming you mean Application Load Balancer/ALB) and NAT Gateway. But with that setup, I could maybe imagine $100/month with on-demand plans. Not $900.
- hayd 5y agoNAT is $1/day and ELB is 60cents/day. So those can't (although those costs excludes data transfer...) be the biggest line items.
- rco8786 5y agoI'll be honest, my experience is like literally the exact opposite. I guess "Once you get good" at anything it seems easy, and learning something else seems like a waste of time. But every time I want to do something in AWS it feels like asking for a house and having someone hand me a pile of bricks and 2x4s and then charging me for 2 houses.
- jjeaff 5y agoYa, but you don't know it's going to cost you as much as two houses until you are done building and start using the house. Then the surprise bills start rolling in.
- jthrowsitaway 5y agoTotally agree. It's been a few years since I used AWS, but back then they shipped half -assed products that didn't have a consistent feel with the rest of their products and weren't feature complete out of the box. I remember using their email service (SES) and wanting to read logs for emails that have been sent. The only option was to point log events at an SQS queue. So, you had to create one of those, assign proper roles and access controls, etc. After that, you need to dump or consume the queue somehow. I can't remember if SQS could dump to S3, but if it could, then OK: create a bucket, assign roles, etc. Now, to parse the S3 bucket contents somehow... Wait, what was I doing? Trying to read some damn log files or set up a Rube Goldberg machine for the log files? I get it. It's a modular system, and that can be very powerful. Sometimes I just want to do something simple without going down a multi-hour (or day) rabbit hole. The inability to perform simple actions on AWS without looping in tons of their other products means indefinite job security for a lot of people. IaC and templates for actions like the above alleviate this a bit, but it's still a dreary landscape.
- rco8786 5y agoYour experience with emails and logs is how I feel every time I try to do anything with AWS as well.
- 4y ago
- simulate-me 5y agoI like AWS, but K8s has reduced my cloud needs to down to a Docker registry and a K8s cluster, which I can easily get anywhere. The one thing I really like about AWS is DynamoDB, which has very cheap and straightforward pricing but can also scale up seamlessly.
- erulabs 5y agoExactly - I use AWS still but it’s more or less EKS and RDS only. Could be anywhere these days - and in fact our ci cluster at linode costs ~1/2 as much and is functionally identical.
- Too 5y agoWhat about storage? Using S3 or some k8s alternative? PV have a very different usage model.
- simulate-me 5y agoI haven't needed to use storage in a serious way. GCS, S3, Digital Ocean storage, they all seem to work relatively the same for my use cases.
- deleted 5y ago[deleted]
- moltar 5y agoTotally agree and this mirrors my experience. I spent years tinkering with devops, custom VPSes, Ansible, etc. AWS CDK is unparalleled experience. Truly a multiplier in productivity that I haven’t experienced in 20 years working on the web.
- VWWHFSfQ 5y agoTerraform was a complete game-changer for me. Once I got good at that suddenly everything became trivial.
- moltar 5y agoYa I can see that. Try CDK though, it’s on the next level. There’s even a cdktf project!
- fdgsdfogijq 5y agoI'm at the point where I'm not sure I will take a dev job on a non AWS stack. My quality of life is just so much higher.
- gregmac 5y agoI'm involved with a project that is built out using colocation space in a traditional datacenter, running on VMware Vsphere. Up to now, they've been doing this very manually: customers assigned to specific, named servers; servers provisioned manually (largely scripted, but not automated); networking configured manually; etc. Worse, they have two deployments in different regions of the world, but they're managed mostly by separate teams and there's a lot of implementation details that are very different. I come from an AWS infrastructure-as-code world, and it's shocking how primitive this stuff all feels. They had considered switching to AWS or Azure in the past, but ultimately deccided to stay on this setup for cost reasons, and frankly I don't think anyone understood the benefits of infrastructure-as-code let alone taking advantage of all the other thing in the cloud providers. I came in to help modernize it. Just as we were getting going with some Terraform-based proof-of-concept, a couple things happened: some sales opportunities came up that would require running in another region, and there were some harder questions on the disaster recovery plan (eg: "what happens if the datacenter burns down?"). The company doesn't want to invest in building out another traditional datacenter, let alone redundant ones, so we took this opportunity to discuss shifting this project over to AWS or Azure. Once built with infrastructure-as-code on the cloud providers a bunch of these problems just disappear, even without making the app itself "cloud-native". Having a dev environment that mirrors production is easy. Having multiple multi-tenant environments in different regions -- and adding new regions -- is also easy. Recovering from a major disaster would also be relatively painless, provided you have a good backup strategy (deploy to new region, restore backups).
- aresant 5y agoI was going to write this same comment. Thinking about the resource cost to prop-up, maintain, and cross-integrate a platform anywhere close to AWS is a business in itself.