4 ms·

I don't think DSL's necessarily fail. Puppet's DSL was everywhere at one point in the late 00's and lots of people argue that Terraform's DSL is fairly ubiquito

I don't think DSL's necessarily fail. Puppet's DSL was everywhere at one point in the late 00's and lots of people argue that Terraform's DSL is fairly ubiquitous in the IaC space. The article brings up some fine points about why DSL's are restrictive, but the primary issue with them comes from the name: They're single use. I meet with companies daily who have tried to adopt Terraform's DSL internally and found a few people feel comfortable with it, but the vast majority of the engineering org have zero interest in learning it because to them - what's the point? They can achieve the same goal with <insert language here> by using Pulumi or CDK. People will bring the flawed argument that Terraform's DSL stops you from footgunning yourself but I've seen plenty of awful, messy Terraform that's only maintainable by the person who wrote it. Another consideration is just how much DSLs evolve over time. You eventually end up getting closer and closer to Turing completeness because people really want to solve problems that require it or solve them in a familiar way. I've spoken frequently about my belief that in the long term, DSLs in the IaC space will largely be superseded by programming language based graph building and most of these thoughts come from frustrated users who are moving away from DSLs

PaulHoule 4y ago
It's amazing how easy it is to do this stuff with conventional programming languages. I wrote a program that built AWS Marketplace images that involved installing a lot of software on a VM, setting up all sorts of resources, building a database for two hours, snapshotting it, etc. This was just a java program that wrote a bash script that did all the work on the remove machine which notified the java program via SQS which then make the image. Since I took the path less taken I got it done quick and if I had to migrate to Azure... easy. If I did it the way most people do it I'd build some Rube Goldberg machine with IaaS that works sometimes and think it was normal.
- nyrikki 4y ago
  Not really surprising that general purpose language that is is targeted at general computing are good at general computing. Puppet was targeted at eventually consistent idempotent configuration changes, mostly as a restricted grammar for a particular use case when distributed clustered systems with shared state were rare. It started to fail once people needed orchestrated stateful changes based on remote state. A task it was never written for. TF's DSL was designed for configuration changes that were ordered and thus to avoid decidability problems enforces a tree like structure on changes exactly how JSON does to avoid cycles, race conditions etc... Except for some who used CFEngine which was also focused around idempotent changes, most of us in the 90's used sed,awk,grep,bash and perl for configuration management. But because we were working on fragile monoliths with limited separation of concerns and modularity....mixed in with freeform code writing that would allow for race conditions, cycles and other issues....the industry decided to limit operations to idempotent tasks. I hand rolled my own OpenStack management code and it was pretty good. But then we couldn't hire people to help us scale that and the company decided to go with Mirantis. Mirantis obviously decided to use Puppet because they liked it or were familiar with it without considering the use case of orchestrating cluster components and Puppet (and chef) was also co-opting meaningful terms like orchestration and rolling out features that really weren't compatible with there original design decisions. While the OpenStack had their own challenges with stability and growing pains, anyone who was a Mirantis customer had very very painful experiences due to them choosing an inappropriate tool, which was actually exceptional for it's original use case. Mirantis's poor design decisions really played a large part in killing most of what was a potential hybrid cloud market, mixed in with companies buying expensive enterprise hardware and SANs. The problem wasn't the DSL. The problem was not making design decisions based on the need of the problem at hand. There is no problem with most DSLs if they are used within the limits of the design choices that were made. If you can't move your systems deployment model to work with TF because you have too much coupling to describe a deployment in a tree...generalized computing languages will not solve your deployment problems because bad architectural decisions were made. A general purpose Turing complete language won't change the fact that being recursively enumerable is required for problems to be semi-decidable. To be clear, I don't view TF or any other existing solution to be perfect. But after years of of helping people move to the cloud, almost universally the problems I see are people either trying to use the wrong product for their need or refusing to adapt there way of working to fit the limitations of the model they chose...and typically chose due to popularity vs the fitness for purpose. As with most things in our industry, the products aren't the problem, the culture is the problem.

PaulHoule 4y ago

It's amazing how easy it is to do this stuff with conventional programming languages. I wrote a program that built AWS Marketplace images that involved installing a lot of software on a VM, setting up all sorts of resources, building a database for two hours, snapshotting it, etc. This was just a java program that wrote a bash script that did all the work on the remove machine which notified the java program via SQS which then make the image. Since I took the path less taken I got it done quick and if I had to migrate to Azure... easy. If I did it the way most people do it I'd build some Rube Goldberg machine with IaaS that works sometimes and think it was normal.

nyrikki 4y ago
Not really surprising that general purpose language that is is targeted at general computing are good at general computing. Puppet was targeted at eventually consistent idempotent configuration changes, mostly as a restricted grammar for a particular use case when distributed clustered systems with shared state were rare. It started to fail once people needed orchestrated stateful changes based on remote state. A task it was never written for. TF's DSL was designed for configuration changes that were ordered and thus to avoid decidability problems enforces a tree like structure on changes exactly how JSON does to avoid cycles, race conditions etc... Except for some who used CFEngine which was also focused around idempotent changes, most of us in the 90's used sed,awk,grep,bash and perl for configuration management. But because we were working on fragile monoliths with limited separation of concerns and modularity....mixed in with freeform code writing that would allow for race conditions, cycles and other issues....the industry decided to limit operations to idempotent tasks. I hand rolled my own OpenStack management code and it was pretty good. But then we couldn't hire people to help us scale that and the company decided to go with Mirantis. Mirantis obviously decided to use Puppet because they liked it or were familiar with it without considering the use case of orchestrating cluster components and Puppet (and chef) was also co-opting meaningful terms like orchestration and rolling out features that really weren't compatible with there original design decisions. While the OpenStack had their own challenges with stability and growing pains, anyone who was a Mirantis customer had very very painful experiences due to them choosing an inappropriate tool, which was actually exceptional for it's original use case. Mirantis's poor design decisions really played a large part in killing most of what was a potential hybrid cloud market, mixed in with companies buying expensive enterprise hardware and SANs. The problem wasn't the DSL. The problem was not making design decisions based on the need of the problem at hand. There is no problem with most DSLs if they are used within the limits of the design choices that were made. If you can't move your systems deployment model to work with TF because you have too much coupling to describe a deployment in a tree...generalized computing languages will not solve your deployment problems because bad architectural decisions were made. A general purpose Turing complete language won't change the fact that being recursively enumerable is required for problems to be semi-decidable. To be clear, I don't view TF or any other existing solution to be perfect. But after years of of helping people move to the cloud, almost universally the problems I see are people either trying to use the wrong product for their need or refusing to adapt there way of working to fit the limitations of the model they chose...and typically chose due to popularity vs the fitness for purpose. As with most things in our industry, the products aren't the problem, the culture is the problem.