3 ms·

I'm really fascinated about why so many infrastructure-as-code tools are actually infrastructure-as-markup. Many of them seem to "evolve" until they're awkward

I'm really fascinated about why so many infrastructure-as-code tools are actually infrastructure-as-markup. Many of them seem to "evolve" until they're awkward attempts at bolting full programming language semantics onto that markup language. I imagine it started out as wanting to keep a simple configuration-style format for infrastructure but fell apart when people's stacks grew large and complex. Projects like Pulumi and CDK seem to be much better approaches, but don't seem to have much traction compared to TF, Cloudformation, etc.

throwaway894345 4y ago
In the early days, people would write imperative scripts to provision infrastructure, but once it's out there you can't just delete resources that you no longer want by deleting the relevant blocks from your script and re-running it--you had to delete them manually or write a "migration script". This was untenable. Then some tools came out which let you use YAML or JSON to describe the desired state of the world, and some tool would diff that against the current state of the world to determine which resources to create, which to delete, and which to update. People started to conflate "YAML" with this sort of diffing tool and they conflated imperative programming languages with the legacy imperative scripting approach. Early infra-as-code vendors capitalized on the YAML=good/programs=bad myth and re-emphasized it by showing cute toy examples and passing off the simplicity as an effect of the YAML/etc rather than the inherent simplicity of the example. Unfortunately, pure YAML/HCL/etc doesn't scale--you end up needing to reference resources from other resources (and/or attributes on those resources), and you end up needing to DRY up many repetitious blocks and so on. What you naturally want is to generate those static YAML configs from a programming language; however, IaC vendors had committed themselves to the "YAML = simple" brand so instead they started building half-assed programming language features into their YAML dialect (effectively using YAML to represent abstract syntax trees for the world's crappiest programming languages). Terraform and CloudFormation fall into this bucket. I guess Helm looked at the landscape and decided it wasn't easy enough to generate syntactically invalid YAML blobs, and consequently decided to use text templates. Eventually the industry caught on to the scam and demanded programming languages back, so we got CDKs which misunderstand the assignment in a different way. Rather than emitting YAML/etc that can be passed into a diff engine, you get some weird bindings that calls (and is called from) a node process (I can't tell what this node process actually does even after reading the docs).
- TheHydroImpulse 4y ago
  CDK outputs CloudFormation templates that you can pass to any other tool if you wish. CDK is technically just a glorified shim over CFN (although 1000x better but you do get caught in the awkwardness that is CFN many times).
  - throwaway894345 4y ago
    It does, but very indirectly which defeats the point of "just emit YAML". I regret phrasing it that way as it seems to have caused a lot of confusion. See my responses to sibling comments for more information.
- happymellon 4y ago
  Amazon's CDK emits YAML. Troposphere emits YAML. You can diff the outputs if you wish, or did I misunderstand your last paragraph.
  - throwaway894345 4y ago
    "Just emit YAML" is about avoiding all of the inheritance and javascript IPC in favor of writing straightforward YAML-builder code in the host language (more like Troposphere, but Troposphere also has poor ergonomics). These are two examples I found online. The first is more complicated but it doesn't do any JavaScript IPC, no inheritance, no mutation, etc. You write it just like you want to write YAML, and it straightforwardly emits YAML (rather than the CDK version which is more opaque/magical). I prefer this: def main(): bucket_name_parameter = ParameterString(Description="The name of the bucket") key_arn_parameter = ParameterString( Description="The ARN of the KMS key used to encrypt the bucket", ) bucket = Bucket(BucketName=bucket_name_parameter) t = Template( description="S3 Bucket Template", parameters={ "BucketName": bucket_name_parameter, "KMSKeyARN": key_arn_parameter, }, resources={ "Bucket": bucket, "BucketPolicy": ManagedPolicy( PolicyDocument={ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowFullAccessToBucket", "Action": "s3:*", "Effect": "Allow", "Resource": Sub( f"${{BucketARN}}/*", BucketARN=bucket.GetArn() ), }, { "Sid": "AllowUseOfTheKey", "Effect": "Allow", "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey", ], "Resource": key_arn_parameter, }, { "Sid": "AllowAttachmentOfPersistentResources", "Effect": "Allow", "Action": [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant", ], "Resource": key_arn_parameter, "Condition": {"Bool": {"kms:GrantIsForAWSResource": True}}, }, ], }, ), }, ) print(json.dumps(t.template_to_cloudformation(), indent=4)) Rather than this: class S3Stack(Stack): def __init__(self, app: App, id: str) -> None: super().__init__(app, id) self.access_point = f"arn:aws:s3:{Aws.REGION}:{Aws.ACCOUNT_ID}:accesspoint/" \ f"{S3_ACCESS_POINT_NAME}" # Set up a bucket bucket = s3.Bucket( self, "example-bucket", access_control=s3.BucketAccessControl.BUCKET_OWNER_FULL_CONTROL, encryption=s3.BucketEncryption.S3_MANAGED, block_public_access=s3.BlockPublicAccess.BLOCK_ALL, ) # Delegating access control to access points # https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-policies.html bucket.add_to_resource_policy( iam.PolicyStatement( actions=["*"], principals=[iam.AnyPrincipal()], resources=[ bucket.bucket_arn, bucket.arn_for_objects('*') ], conditions={ "StringEquals": { "s3:DataAccessPointAccount": f"{Aws.ACCOUNT_ID}" } } ), ) Note: Compared to Troposphere, the bindings in the first example are completely generated from a spec published by AWS, so they never fall behind. They're also type-annotated so you can use those bindings with type safety. Sadly, the project has been abandoned because it's CloudFormation-specific and the world has moved away from CloudFormation.
wolfd 4y ago
I am fond of Starlark, the Python-like configuration language of Bazel. It's code, but they've removed most of the footguns that you encounter when using Python as configuration (e.g. you can't mutate global state). I think it's a pleasant middle ground between config and code, it's definitely code, but limited.

throwaway894345 4y ago

In the early days, people would write imperative scripts to provision infrastructure, but once it's out there you can't just delete resources that you no longer want by deleting the relevant blocks from your script and re-running it--you had to delete them manually or write a "migration script". This was untenable. Then some tools came out which let you use YAML or JSON to describe the desired state of the world, and some tool would diff that against the current state of the world to determine which resources to create, which to delete, and which to update. People started to conflate "YAML" with this sort of diffing tool and they conflated imperative programming languages with the legacy imperative scripting approach. Early infra-as-code vendors capitalized on the YAML=good/programs=bad myth and re-emphasized it by showing cute toy examples and passing off the simplicity as an effect of the YAML/etc rather than the inherent simplicity of the example. Unfortunately, pure YAML/HCL/etc doesn't scale--you end up needing to reference resources from other resources (and/or attributes on those resources), and you end up needing to DRY up many repetitious blocks and so on. What you naturally want is to generate those static YAML configs from a programming language; however, IaC vendors had committed themselves to the "YAML = simple" brand so instead they started building half-assed programming language features into their YAML dialect (effectively using YAML to represent abstract syntax trees for the world's crappiest programming languages). Terraform and CloudFormation fall into this bucket. I guess Helm looked at the landscape and decided it wasn't easy enough to generate syntactically invalid YAML blobs, and consequently decided to use text templates. Eventually the industry caught on to the scam and demanded programming languages back, so we got CDKs which misunderstand the assignment in a different way. Rather than emitting YAML/etc that can be passed into a diff engine, you get some weird bindings that calls (and is called from) a node process (I can't tell what this node process actually does even after reading the docs).

TheHydroImpulse 4y ago
CDK outputs CloudFormation templates that you can pass to any other tool if you wish. CDK is technically just a glorified shim over CFN (although 1000x better but you do get caught in the awkwardness that is CFN many times).
- throwaway894345 4y ago
  It does, but very indirectly which defeats the point of "just emit YAML". I regret phrasing it that way as it seems to have caused a lot of confusion. See my responses to sibling comments for more information.
happymellon 4y ago
Amazon's CDK emits YAML. Troposphere emits YAML. You can diff the outputs if you wish, or did I misunderstand your last paragraph.
- throwaway894345 4y ago
  "Just emit YAML" is about avoiding all of the inheritance and javascript IPC in favor of writing straightforward YAML-builder code in the host language (more like Troposphere, but Troposphere also has poor ergonomics). These are two examples I found online. The first is more complicated but it doesn't do any JavaScript IPC, no inheritance, no mutation, etc. You write it just like you want to write YAML, and it straightforwardly emits YAML (rather than the CDK version which is more opaque/magical). I prefer this: def main(): bucket_name_parameter = ParameterString(Description="The name of the bucket") key_arn_parameter = ParameterString( Description="The ARN of the KMS key used to encrypt the bucket", ) bucket = Bucket(BucketName=bucket_name_parameter) t = Template( description="S3 Bucket Template", parameters={ "BucketName": bucket_name_parameter, "KMSKeyARN": key_arn_parameter, }, resources={ "Bucket": bucket, "BucketPolicy": ManagedPolicy( PolicyDocument={ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowFullAccessToBucket", "Action": "s3:*", "Effect": "Allow", "Resource": Sub( f"${{BucketARN}}/*", BucketARN=bucket.GetArn() ), }, { "Sid": "AllowUseOfTheKey", "Effect": "Allow", "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey", ], "Resource": key_arn_parameter, }, { "Sid": "AllowAttachmentOfPersistentResources", "Effect": "Allow", "Action": [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant", ], "Resource": key_arn_parameter, "Condition": {"Bool": {"kms:GrantIsForAWSResource": True}}, }, ], }, ), }, ) print(json.dumps(t.template_to_cloudformation(), indent=4)) Rather than this: class S3Stack(Stack): def __init__(self, app: App, id: str) -> None: super().__init__(app, id) self.access_point = f"arn:aws:s3:{Aws.REGION}:{Aws.ACCOUNT_ID}:accesspoint/" \ f"{S3_ACCESS_POINT_NAME}" # Set up a bucket bucket = s3.Bucket( self, "example-bucket", access_control=s3.BucketAccessControl.BUCKET_OWNER_FULL_CONTROL, encryption=s3.BucketEncryption.S3_MANAGED, block_public_access=s3.BlockPublicAccess.BLOCK_ALL, ) # Delegating access control to access points # https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-policies.html bucket.add_to_resource_policy( iam.PolicyStatement( actions=["*"], principals=[iam.AnyPrincipal()], resources=[ bucket.bucket_arn, bucket.arn_for_objects('*') ], conditions={ "StringEquals": { "s3:DataAccessPointAccount": f"{Aws.ACCOUNT_ID}" } } ), ) Note: Compared to Troposphere, the bindings in the first example are completely generated from a spec published by AWS, so they never fall behind. They're also type-annotated so you can use those bindings with type safety. Sadly, the project has been abandoned because it's CloudFormation-specific and the world has moved away from CloudFormation.

wolfd 4y ago

I am fond of Starlark, the Python-like configuration language of Bazel. It's code, but they've removed most of the footguns that you encounter when using Python as configuration (e.g. you can't mutate global state). I think it's a pleasant middle ground between config and code, it's definitely code, but limited.