3 ms·

To my surprise this is mostly solid advice despite the clickbaity title. I've never used Cognito or ElasticCache, but for the other three (CloudFormation, Kine

To my surprise this is mostly solid advice despite the clickbaity title. I've never used Cognito or ElasticCache, but for the other three (CloudFormation, Kinesis, and Lambda), I would agree with what he's actually saying, which is to use these services judiciously. CloudFormation has some advantages over Terraform, but having used both, Terraform is much more usable for long-lived static resources. I have yet to try Pulumi, and I don't see any reason why I should to be honest. However, CloudFormation is required for automated build and deployment of Lambda functions--both Serverless Framework and AWS SAM use CloudFormation. I've also used CloudFormation StackSets to build out infrastructure for multi-account governance. In normal use though, Terraform is typically more usable, though both solutions can get pretty hairy. By "Kinesis" the author seems to be referring to Kinesis Streams. In fairness I haven't built anything where Kinesis Streams would be a better use case than SNS/SQS, but that's just as much a statement about the projects I have happened to work on than it is about Kinesis Streams. I have used Kinesis Firehose, which provides a very scalable mechanism for, "multiple clients are intermittently spitting out a massive volume of data points, all of which need to be faithfully logged into S3 somewhere". And finally, we come to Lambda. Lambda is a good fit for use cases where either you need to run some code on an intermittent basis or when you want to prototype faster and don't want to mess around with provisioning and deploying to servers. Lambda is a great place for little pieces of glue code that get triggered on a x-minute cron or based on a CloudWatch event or from SNS or SQS. It's good for serving an HTTP API that gets less than one request per second. These use cases are extremely common and I've encountered them a lot more than the author seems to, though again maybe that's just me. But for anything high traffic, where the Lambda container is just going to stay hot, just deploy it to EC2 (perhaps with some container magic in the middle), it's cheaper in the long run.

x3n0ph3n3 7y ago
My team has invested a sizable amount of code to manage CloudFormation. At this point, we have a rather mature interface for dealing with CloudFormation, including: - defining dependencies between stacks - taking outputs from stacks and feeding them in as parameters to other stacks (not using that awful Import/Export Value crap they implemented) - deploying as many stacks in parallel as possible and waiting for them to complete before deploying dependent stacks - dealing with common failures and rollbacks, including handling known "continue update rollback" steps with predefined resources to skip - pre/post stack create/update/delete actions to make API calls and perform other actions outside of CloudFormation We basically built the missing pieces of CloudFormation ourselves and have managed to keep CloudFormation holding the definitive state rather than managing it ourselves (or paying Terraform to do it). We have about 500 stacks for a single deployment of our product. We also implemented a large number of Transform Macro lambdas to composable templates _much_ easier. There's a lot you can do with CloudFormation, but it takes some investment. A few things about it that really drive me nuts, though: - Lagging support for new features/resources - Parameter count and parameter size limits - Certain bugs with some resource types that are slow to get fixed. (Redshift cluster password management and issues with elasticsearch resizing triggering blue/green deployment come to mind. - No ability to modify timeouts for some actions. For example, the timeout for a CustomResource is fixed and cannot be tuned -- if your Lambda never responds, CloudFormation will hang up to 2 hours. We wrote our own Lambda wrapper just to guarantee a response if an unexpected failure occurs
x3n0ph3n3 7y ago
My team has a case with Kinesis streams where multiple lambdas (and a firehose) processes the same data and performs different actions on them. This can't be done with SQS, and there's no "pausing" with SNS. If for some reason, a lambda has issues processing data, we have until the TRIM_HORIZON to fix the issue and resume processing. I will say, though, the the KCL library is a PoS and the few containerize services that are reading from Kinesis exhibit this annoying behavior of causing the iterator age to spike to the TRIM_HORIZON when performing a blue/green deployment for the containers.

x3n0ph3n3 7y ago

My team has invested a sizable amount of code to manage CloudFormation. At this point, we have a rather mature interface for dealing with CloudFormation, including: - defining dependencies between stacks - taking outputs from stacks and feeding them in as parameters to other stacks (not using that awful Import/Export Value crap they implemented) - deploying as many stacks in parallel as possible and waiting for them to complete before deploying dependent stacks - dealing with common failures and rollbacks, including handling known "continue update rollback" steps with predefined resources to skip - pre/post stack create/update/delete actions to make API calls and perform other actions outside of CloudFormation We basically built the missing pieces of CloudFormation ourselves and have managed to keep CloudFormation holding the definitive state rather than managing it ourselves (or paying Terraform to do it). We have about 500 stacks for a single deployment of our product. We also implemented a large number of Transform Macro lambdas to composable templates _much_ easier. There's a lot you can do with CloudFormation, but it takes some investment. A few things about it that really drive me nuts, though: - Lagging support for new features/resources - Parameter count and parameter size limits - Certain bugs with some resource types that are slow to get fixed. (Redshift cluster password management and issues with elasticsearch resizing triggering blue/green deployment come to mind. - No ability to modify timeouts for some actions. For example, the timeout for a CustomResource is fixed and cannot be tuned -- if your Lambda never responds, CloudFormation will hang up to 2 hours. We wrote our own Lambda wrapper just to guarantee a response if an unexpected failure occurs

x3n0ph3n3 7y ago

My team has a case with Kinesis streams where multiple lambdas (and a firehose) processes the same data and performs different actions on them. This can't be done with SQS, and there's no "pausing" with SNS. If for some reason, a lambda has issues processing data, we have until the TRIM_HORIZON to fix the issue and resume processing. I will say, though, the the KCL library is a PoS and the few containerize services that are reading from Kinesis exhibit this annoying behavior of causing the iterator age to spike to the TRIM_HORIZON when performing a blue/green deployment for the containers.