6 ms·

The Snowflakes as Code Antipattern

36 points by zdw 5y ago

mrhands556 5y ago
Wait… different branches for different environments? Wouldn’t you run into the same problems?
- officialchicken 5y ago
  Not really, but it depends on what the repo represents. Not all envs can be identical despite the logical preference - non-wildcard SSL certs for example. I think the author here is referring to an infrastructure repo with branch-per-env as a common pattern for devops / IaaC / terraform code bases, versus env-per-folder or similar in the trunk if you absolutely have to have differences between envs. It mentions deploying build artifacts from a parallel CI/CD at the end of the article for running the software running on the infra.
- jb_s 5y ago
  Yes. I personally prefer having IaC in the same repo as the code. Any changes to the code OR the IaC uses the git flow as per normal and requires a PR. When the PR is merged to develop its pushed to QA environments automatically - therefore updating infrastructure with the updated IaC. We update deployment configuration separately (because why would we include anything environment-specific in the repo?), this never presents any major issues although sometimes I guess you'd need to update the config just at the same time as merging the code PR which is slightly inconvenient and is somewhat OCD triggering, but its a productive setup so I don't mind.
  - steveBK123 5y ago
    My orgs problem is that the IaC is owned by different people than the apps code running on the infra the IaC controls. Much fun ensues in trying to have any sense of a predictable timeline when you want to rollout any app feature requiring IaC changes. Feels worse than on-prem datacenter work cuz at least there the team was bigger, and it was slow, but mature with a ticketing system, SLAs and no key man dependencies. With someone-else-owns-myIaC, I am dealing with 2-3 DevOps dudes who kind of sort of use Jira sometimes, and lack staff in some of the region timezones our business operates in.
rightbyte 5y ago
I got tired reading that. How about letting me just push php files via ftp to some outdated Debian machine. All these layers of complexity made me swear never to touch "devops" again.
- 3np 5y ago
  The way I read the article, the “code” being talked about is not necessarily application code but the IaC used to provision the actual infrastructure. That is, the target audience is the sysadmin who would provision the PHP/FTP server and discussing how to sanely manage that as the fleet grows with slightly separate but mostly similar differences between instances. Maybe you as a web developer don’t want this complexity but a lot of it is inherent and someone has to deal with it at the end of the day.
  - cyanydeez 5y ago
    i read it as semantic satiation. all hes done is change the terms, but you still have snowflakes because some system somewhere is cofiguring your snosflakes.
    - joshuamorton 5y ago
      If the snowflake is defined as class SnowflakeConfig(NormalConfig): # Normally this is 6, but for staging it has to be 2 foo = 2 and then you have something like deployments: us-east-1.prod: NormalConfig us-east-1.staging = SnowflakeConfig is it really the same? Granted, I don't think the article actually suggests that as a solution, so I don't think it provides a great response.
- Spivak 5y ago
  I mean modern devops is basically circling back around to that idea but with “php files” replaced with “git repo.” To deploy code to production at $dayjob all that’s necessary is to commit to master and press the play button once you’re satisfied the code works in dev/qa/staging. The pipeline is relatively complex because production ready infrastructure has some irreducible complexity but devs don’t have to care about it most of the time. In some cases we adopt a team of devs to work on infra related performance initiatives but that’s rare.
scubbo 5y ago
Maybe I'm spoiled, but this seems like a solution in search of a problem. Is it truly common to have separate environments described by independent copy-pastes of the conceptually-same code, rather than one configurable CDK construct or Terraform equivalent? If I heard "When someone makes a change to the code for one instance, they copy or merge the change to other instances. The process for doing this is usually manual, and involves effort and care to ensure that deliberate differences between instances are maintained, while avoiding unintended differences." rather than using a deployment pipeline, I'd run screaming.
- 3np 5y ago
  You can still end up here with a stellar deployment pipeline. Let’s say you have servers on AWS, Scaleway and OVH, running separate instances of the same code, in staging/test/prod. They don’t provide the exact same abstractions and accesses for VPCs and VMs. Certainly not for PaaS workloads like containers.
  - scubbo 5y ago
    Thanks for explaining! I appreciate the insight. Seem that I _have_ indeed, been spoiled (or, at least, coddled), because I've never worked on a multi-cloud environment, so have never had to consider such a case.
- edgan 5y ago
  Terraform leads to this pattern. Unless you can make your environments exact mirror images of each other, this is what you end up with. Almost no one does exact mirror images for cost reasons. Every environment would have to be at production scale.
  - scubbo 5y ago
    The "this" in your second paragraph has an unclear referent. Are you saying that copy-pasted code definitions per-environment are necessary unless those environments are exact mirror-images? If so, that is untrue - CDK (and, I assume, any other IaC framework, with the appropriate terminology change) allows Constructs to be instantiated with different variables. For instance, you could have a Construct for your environment that accepts a `numServers` variable, and instantiate that Construct multiple times with different variables for beta, gamma, etc. Only one instance of the definitional code, but multiple instantiations.
andrekandre 5y ago
> Other teams use a folder structure to maintain separate projects for each environment. They copy and edit code between projects to make changes across environments. if i'm reading this right, they're not saying different folders with only the diffs (config files, test classes, dummy data, etc) but actually full-on copies of the whole entire project?
- 3np 5y ago
  Yeah, it was a common pattern especially in older TF repositories from before workspaces had better support. It takes quite a bit of discipline (and balance; sometimes the effort to make it “right” is not well-spent) to not end up in a mess when your infrastructure starts to scale across regions and providers.
  - 0x202020 5y ago
    To get around the separate directories/copies someone established a pattern of doing a for each environment at the top of each tf file and then all variables are stored as a map of each env, which I find interesting. The pattern exists in one repo with the other being all the duplicates, they’ve been in “it just works” for long enough that no one has changed them. Workspaces look pretty slick, I’ll have to check them out
  - edgan 5y ago
    I would say it is still a common pattern. I don't see how workspaces really improves anything. The fundamental problem with Terraform is it lacks if statements. There are solutions that involve dynamic blocks and for_each, but I don't think they completely solve the problem. I would say they don't have great readability. I would love to hear a reasonable way to not snowflake that doesn't dive down the dynamic hole. Overall I think this is all an argument for Pulumi over Terraform.
tyingq 5y ago
>The two main ways people implement snowflakes as code are folders and branches. I've seen plenty of yaml and/or json used for this, though maybe he's saying that's the sort of thing that isn't a snowflake? But it's just basically a folder structure in "code".
edgan 5y ago
He has a real point, but it doesn't really give a good alternative other than "read my book". The crux of the problem when writing Terraform code is that it doesn't have if statements. Without them you end up doing hacky things to try to recreate them. Environments are not all going to be exact replicas of each other for 100 different reasons, and so trying to use the exact same code for each just doesn't work. I don't consider dynamic/for_each an elegant solution with the current syntax.
- jb_s 5y ago
  Haven't used Terraform - ARM templates have a workable (though not elegant) expression system where you can use parameters to resolve stuff - so you indeed have one IaC definition and use your CI/CD to set parameter values and therefore setup whatever is needed for different environments. So your CI/CD knows about the envs and not the IaC code. As it should be I think.
pjc50 5y ago
I thought this was going to be more literal: https://flatline.org.uk/snowflake.pl.txt https://flatline.org.uk/snowflake.pl.txt $w= 100 ;$w*=5; ;$W=$w* $w; q--; $p=atan2(1,0)* 2;3; @P= ("\000"x3)x( $W); BEGIN{$";sub S{$_.='fhoq' }sub K{$_.='s'.(( $n ++)) .(( '' .'' .'{' )) ;}; sub Y{$_ .= $_[0 ]}; S;K ;Y(''. '$k/=2;$'. '' .'l/=2}'); S;K;0; Y( '$k/=3;$l/=3;$k+=$j/4}fhoqh{'. '' .'cevq"P6\a$jq$j\a255\a",q@P}');S;K;Y('$g=$c/3;e}'); S;K; Y(q.$g=$. .'c/-3;' .'e}');S ;K;Y('$'. 'g'. '=' .'$c*2' .'/3}' );;s/v /vag/x; sub p{$P [$x+$w/2+ int($y)* $w+$W/2] ="\377"x3 }sub r{$z=$x*cos($t)+$y*sin($t);$y=$x*sin($t)-$y*cos($t); $x =$z};tr/a-z/n-za-m/;s/d/$"/gx; $x ++;$y= 4;eval$_}; s/ /japh/;sub T{;$a= $_[ 0]; ;$b= (( $a&( ($a |(+ $a <<1) )<< 1) )>> 2)|( $a >>3);if($b){ $_[0 ]=""}else{$_ [0]="f$a;p;" }};; for(1..20000 *10 ){$_ =int(rand(20)) ;T$_ ;0; eval$_} u#byPJC #JA PH# (archive of more snowflake perl: https://web.archive.org/web/20080515091339/http://cam.pm.org/christmasppc-summary.shtml https://web.archive.org/web/20080515091339/http://cam.pm.org... )

36 points by zdw 5y ago

mrhands556 5y ago

Wait… different branches for different environments? Wouldn’t you run into the same problems?

officialchicken 5y ago
Not really, but it depends on what the repo represents. Not all envs can be identical despite the logical preference - non-wildcard SSL certs for example. I think the author here is referring to an infrastructure repo with branch-per-env as a common pattern for devops / IaaC / terraform code bases, versus env-per-folder or similar in the trunk if you absolutely have to have differences between envs. It mentions deploying build artifacts from a parallel CI/CD at the end of the article for running the software running on the infra.
jb_s 5y ago
Yes. I personally prefer having IaC in the same repo as the code. Any changes to the code OR the IaC uses the git flow as per normal and requires a PR. When the PR is merged to develop its pushed to QA environments automatically - therefore updating infrastructure with the updated IaC. We update deployment configuration separately (because why would we include anything environment-specific in the repo?), this never presents any major issues although sometimes I guess you'd need to update the config just at the same time as merging the code PR which is slightly inconvenient and is somewhat OCD triggering, but its a productive setup so I don't mind.
- steveBK123 5y ago
  My orgs problem is that the IaC is owned by different people than the apps code running on the infra the IaC controls. Much fun ensues in trying to have any sense of a predictable timeline when you want to rollout any app feature requiring IaC changes. Feels worse than on-prem datacenter work cuz at least there the team was bigger, and it was slow, but mature with a ticketing system, SLAs and no key man dependencies. With someone-else-owns-myIaC, I am dealing with 2-3 DevOps dudes who kind of sort of use Jira sometimes, and lack staff in some of the region timezones our business operates in.

rightbyte 5y ago

I got tired reading that. How about letting me just push php files via ftp to some outdated Debian machine. All these layers of complexity made me swear never to touch "devops" again.

3np 5y ago
The way I read the article, the “code” being talked about is not necessarily application code but the IaC used to provision the actual infrastructure. That is, the target audience is the sysadmin who would provision the PHP/FTP server and discussing how to sanely manage that as the fleet grows with slightly separate but mostly similar differences between instances. Maybe you as a web developer don’t want this complexity but a lot of it is inherent and someone has to deal with it at the end of the day.
- cyanydeez 5y ago
  i read it as semantic satiation. all hes done is change the terms, but you still have snowflakes because some system somewhere is cofiguring your snosflakes.
  - joshuamorton 5y ago
    If the snowflake is defined as class SnowflakeConfig(NormalConfig): # Normally this is 6, but for staging it has to be 2 foo = 2 and then you have something like deployments: us-east-1.prod: NormalConfig us-east-1.staging = SnowflakeConfig is it really the same? Granted, I don't think the article actually suggests that as a solution, so I don't think it provides a great response.
Spivak 5y ago
I mean modern devops is basically circling back around to that idea but with “php files” replaced with “git repo.” To deploy code to production at $dayjob all that’s necessary is to commit to master and press the play button once you’re satisfied the code works in dev/qa/staging. The pipeline is relatively complex because production ready infrastructure has some irreducible complexity but devs don’t have to care about it most of the time. In some cases we adopt a team of devs to work on infra related performance initiatives but that’s rare.

scubbo 5y ago

Maybe I'm spoiled, but this seems like a solution in search of a problem. Is it truly common to have separate environments described by independent copy-pastes of the conceptually-same code, rather than one configurable CDK construct or Terraform equivalent? If I heard "When someone makes a change to the code for one instance, they copy or merge the change to other instances. The process for doing this is usually manual, and involves effort and care to ensure that deliberate differences between instances are maintained, while avoiding unintended differences." rather than using a deployment pipeline, I'd run screaming.

3np 5y ago
You can still end up here with a stellar deployment pipeline. Let’s say you have servers on AWS, Scaleway and OVH, running separate instances of the same code, in staging/test/prod. They don’t provide the exact same abstractions and accesses for VPCs and VMs. Certainly not for PaaS workloads like containers.
- scubbo 5y ago
  Thanks for explaining! I appreciate the insight. Seem that I _have_ indeed, been spoiled (or, at least, coddled), because I've never worked on a multi-cloud environment, so have never had to consider such a case.
edgan 5y ago
Terraform leads to this pattern. Unless you can make your environments exact mirror images of each other, this is what you end up with. Almost no one does exact mirror images for cost reasons. Every environment would have to be at production scale.
- scubbo 5y ago
  The "this" in your second paragraph has an unclear referent. Are you saying that copy-pasted code definitions per-environment are necessary unless those environments are exact mirror-images? If so, that is untrue - CDK (and, I assume, any other IaC framework, with the appropriate terminology change) allows Constructs to be instantiated with different variables. For instance, you could have a Construct for your environment that accepts a `numServers` variable, and instantiate that Construct multiple times with different variables for beta, gamma, etc. Only one instance of the definitional code, but multiple instantiations.

andrekandre 5y ago

> Other teams use a folder structure to maintain separate projects for each environment. They copy and edit code between projects to make changes across environments. if i'm reading this right, they're not saying different folders with only the diffs (config files, test classes, dummy data, etc) but actually full-on copies of the whole entire project?

3np 5y ago
Yeah, it was a common pattern especially in older TF repositories from before workspaces had better support. It takes quite a bit of discipline (and balance; sometimes the effort to make it “right” is not well-spent) to not end up in a mess when your infrastructure starts to scale across regions and providers.
- 0x202020 5y ago
  To get around the separate directories/copies someone established a pattern of doing a for each environment at the top of each tf file and then all variables are stored as a map of each env, which I find interesting. The pattern exists in one repo with the other being all the duplicates, they’ve been in “it just works” for long enough that no one has changed them. Workspaces look pretty slick, I’ll have to check them out
- edgan 5y ago
  I would say it is still a common pattern. I don't see how workspaces really improves anything. The fundamental problem with Terraform is it lacks if statements. There are solutions that involve dynamic blocks and for_each, but I don't think they completely solve the problem. I would say they don't have great readability. I would love to hear a reasonable way to not snowflake that doesn't dive down the dynamic hole. Overall I think this is all an argument for Pulumi over Terraform.

tyingq 5y ago

>The two main ways people implement snowflakes as code are folders and branches. I've seen plenty of yaml and/or json used for this, though maybe he's saying that's the sort of thing that isn't a snowflake? But it's just basically a folder structure in "code".

edgan 5y ago

He has a real point, but it doesn't really give a good alternative other than "read my book". The crux of the problem when writing Terraform code is that it doesn't have if statements. Without them you end up doing hacky things to try to recreate them. Environments are not all going to be exact replicas of each other for 100 different reasons, and so trying to use the exact same code for each just doesn't work. I don't consider dynamic/for_each an elegant solution with the current syntax.

jb_s 5y ago
Haven't used Terraform - ARM templates have a workable (though not elegant) expression system where you can use parameters to resolve stuff - so you indeed have one IaC definition and use your CI/CD to set parameter values and therefore setup whatever is needed for different environments. So your CI/CD knows about the envs and not the IaC code. As it should be I think.

pjc50 5y ago

I thought this was going to be more literal: https://flatline.org.uk/snowflake.pl.txt https://flatline.org.uk/snowflake.pl.txt $w= 100 ;$w*=5; ;$W=$w* $w; q--; $p=atan2(1,0)* 2;3; @P= ("\000"x3)x( $W); BEGIN{$";sub S{$_.='fhoq' }sub K{$_.='s'.(( $n ++)) .(( '' .'' .'{' )) ;}; sub Y{$_ .= $_[0 ]}; S;K ;Y(''. '$k/=2;$'. '' .'l/=2}'); S;K;0; Y( '$k/=3;$l/=3;$k+=$j/4}fhoqh{'. '' .'cevq"P6\a$jq$j\a255\a",q@P}');S;K;Y('$g=$c/3;e}'); S;K; Y(q.$g=$. .'c/-3;' .'e}');S ;K;Y('$'. 'g'. '=' .'$c*2' .'/3}' );;s/v /vag/x; sub p{$P [$x+$w/2+ int($y)* $w+$W/2] ="\377"x3 }sub r{$z=$x*cos($t)+$y*sin($t);$y=$x*sin($t)-$y*cos($t); $x =$z};tr/a-z/n-za-m/;s/d/$"/gx; $x ++;$y= 4;eval$_}; s/ /japh/;sub T{;$a= $_[ 0]; ;$b= (( $a&( ($a |(+ $a <<1) )<< 1) )>> 2)|( $a >>3);if($b){ $_[0 ]=""}else{$_ [0]="f$a;p;" }};; for(1..20000 *10 ){$_ =int(rand(20)) ;T$_ ;0; eval$_} u#byPJC #JA PH# (archive of more snowflake perl: https://web.archive.org/web/20080515091339/http://cam.pm.org/christmasppc-summary.shtml https://web.archive.org/web/20080515091339/http://cam.pm.org... )