3 ms·

While I understand the sentiment, I don't really get the particulars. So, instead of learning Puppet or Terraform, you want to implement one from scratch? I h

While I understand the sentiment, I don't really get the particulars. So, instead of learning Puppet or Terraform, you want to implement one from scratch? I have that itch too, I also try to implement the basics of something like git or Ansible in my own toy system when learning it. But that's just to get some hang of why the tool looks like it does. I'm still there to use the tool. That goes double if you're working with others. I totally get the bit about wanting to learn fundamentals, but not the bit how it turns you off some higher level tool.

g9yuayon 4y ago
Not really. Use whatever technology that you know best, but hide unnecessary complexity through the right tools. I'm not sure about the OP, but I had extremely good experience when I was in Netflix. A single engineer, Joe Sondow, created the app Asgard. Using Asgard, I didn't need anyone to teach me how to configure clusters or networking topologies. The app got affordance, consistency, and constraints right, to the point that at any point what I guessed was either right or the only way to complete my tasks. Michael from the cloud team created command-line tools for managing AMIs. They exposed just what one needs to achieve while hiding anything about how things were done. As a result, I can still remember how to use these tools, years after I left Netflix. Similarly for doing service discovery, fault injection, load balancing, autoscaling, predictive autoscaling, logging events, publishing metrics, and the list can go on. Everything in Netflix, at least when I was there, followed the unspoken principle of making everything a non-event: zero unnecessary mental load, maximum flexibility, maximum freedom to engineers, and full transparency. Things just happened. Tools were just in place. The tools could be primitive and could lack of features, but the they did not get in my way whatsoever.
- fivre 4y ago
  > [The AMI management tools] exposed just what one needs to achieve while hiding anything about how things were done No, they exposed just what _someone working at Netflix who produces AMIs for Netflix's infrastructure and is required to conform to assorted Netflix expectations lest they be told to go pound sand_ needs. Accepting that constraint makes the problem much easier. When you try to make a general-purpose tool, you rapidly run up against: - Some other user needs to do something that you don't, that the existing configuration surface cannot possibly account for. You now have either a tool that exposes two (and soon, three or four or five ad infinitum) means of doing thing (likely without any indication why you'd use one or the other) or a tool that exposes <original means of doing thing> and <fill in your own thing from scratch>. - Someone has other parts of their infrastructure whose design is, for lack of a better description, "fucking stupid". They don't have the power or will to change that, but they're happy to ask you to include a workaround for it in your product. They also have a number of friends whose surrounding infrastructure is similarly fucking stupid, and some of those are important contracts where sales will force your hand to build the workaround instead of asking the customer not do fucking stupid things. Yes, a single well-managed organization can delegate various pieces of complexity throughout the org and avoid having other parts worry about them. General-purpose tools do not have that complexity. The reason you have to learn new concepts and tools to learn Terraform effectively is because you're taking all the complexity of dealing with a cloud services provider and then adding an additional layer of complexity on top to augment the things you can do with the cloud provider. The value proposition is that this still better than what you'd do by hand and, crucially, something that other people who know Terraform can understand. Realistically, you can hire another engineer who's familiar with Terraform. You can't clone Matt.
  - deleted 4y ago
    [deleted]
- xorcist 4y ago
  The reasoning above perhaps isn't so much related to your use of Sondow's tools, but how Sondow and his colleagues develops and maintains those tools.
dekhn 4y ago
I wrote my own Ansible+terraform alternative and frankly, for the limited use (demo of GCE launch, we could start over 1000 raw vms and be doing large scale compute in minutes during the live demo (unfortunately urs wasn't confident so we pretaped that part). Ansible is not bad but tf is Terrible. Good idea, terrible execution and evolution
philliphaydon 4y ago
> So, instead of learning Puppet or Terraform, you want to implement one from scratch? I definitely don't want to implement from scratch. I share his sentiments. I keep hearing about Puppet Labs (i think is the right one), but I've been to the site a bunch of times and I end up leaving because I have no idea where to start. So I really donno how I would benefit over just using a few bash scripts or powershell scripts to manage my Linux/Windows servers inn AWS. Many different things are like this. The barrier to entry is high, once you get started it becomes easier and you tend to agree that the tool is much better than other alternatives. However I today learned that Pinia is the state management library that replaces Vuex in Vue. (effectively V5 of Vuex) and I opened up the documentation, and within a few minutes of reading I had it up in running in my little side project. The documentation explained why its now Pinia, and gave a concrete working example of a simple counter and how its used. Vue, Rust, Postgres - these all have my favourite documentation.
- randomswede 4y ago
  If a few bash scripts is enough for you to manage your servers, you probably don't need puppet (or ansible, or chef). But, yes, I agree that the puppet documentation is (or at least was, I have not looked at it for several years) not entirely brilliant. It is an OK reference manual, but like so much documentation, it is more difficult than necessary for learning the thing.
ratww 4y ago
I don't understand how you were able to make the jump to "I don't like tools that lie about complexity" to "I want to build tools from scratch", but I share GP's complaints. The problem is clearly with tools that promise simplicity for users but are actually only simple for the creators of the tools themselves. For end users there's still a lot of complexity. And it's not that complexity is a problem per se. It's just that it exists on multiple levels. And some of those tools aren't abstracting much or anything at those levels. They're often just providing yet another thin API for something that's still complex for their users, so they haven't really made the hard thing simple, maybe they simplified the 10% of it that don't matter at the user's scale. If anything they introduced even more complexity. Aka, leaky abstractions. Of course, YMMV.

g9yuayon 4y ago

Not really. Use whatever technology that you know best, but hide unnecessary complexity through the right tools. I'm not sure about the OP, but I had extremely good experience when I was in Netflix. A single engineer, Joe Sondow, created the app Asgard. Using Asgard, I didn't need anyone to teach me how to configure clusters or networking topologies. The app got affordance, consistency, and constraints right, to the point that at any point what I guessed was either right or the only way to complete my tasks. Michael from the cloud team created command-line tools for managing AMIs. They exposed just what one needs to achieve while hiding anything about how things were done. As a result, I can still remember how to use these tools, years after I left Netflix. Similarly for doing service discovery, fault injection, load balancing, autoscaling, predictive autoscaling, logging events, publishing metrics, and the list can go on. Everything in Netflix, at least when I was there, followed the unspoken principle of making everything a non-event: zero unnecessary mental load, maximum flexibility, maximum freedom to engineers, and full transparency. Things just happened. Tools were just in place. The tools could be primitive and could lack of features, but the they did not get in my way whatsoever.

fivre 4y ago
> [The AMI management tools] exposed just what one needs to achieve while hiding anything about how things were done No, they exposed just what _someone working at Netflix who produces AMIs for Netflix's infrastructure and is required to conform to assorted Netflix expectations lest they be told to go pound sand_ needs. Accepting that constraint makes the problem much easier. When you try to make a general-purpose tool, you rapidly run up against: - Some other user needs to do something that you don't, that the existing configuration surface cannot possibly account for. You now have either a tool that exposes two (and soon, three or four or five ad infinitum) means of doing thing (likely without any indication why you'd use one or the other) or a tool that exposes <original means of doing thing> and <fill in your own thing from scratch>. - Someone has other parts of their infrastructure whose design is, for lack of a better description, "fucking stupid". They don't have the power or will to change that, but they're happy to ask you to include a workaround for it in your product. They also have a number of friends whose surrounding infrastructure is similarly fucking stupid, and some of those are important contracts where sales will force your hand to build the workaround instead of asking the customer not do fucking stupid things. Yes, a single well-managed organization can delegate various pieces of complexity throughout the org and avoid having other parts worry about them. General-purpose tools do not have that complexity. The reason you have to learn new concepts and tools to learn Terraform effectively is because you're taking all the complexity of dealing with a cloud services provider and then adding an additional layer of complexity on top to augment the things you can do with the cloud provider. The value proposition is that this still better than what you'd do by hand and, crucially, something that other people who know Terraform can understand. Realistically, you can hire another engineer who's familiar with Terraform. You can't clone Matt.
- deleted 4y ago
  [deleted]
xorcist 4y ago
The reasoning above perhaps isn't so much related to your use of Sondow's tools, but how Sondow and his colleagues develops and maintains those tools.

dekhn 4y ago

I wrote my own Ansible+terraform alternative and frankly, for the limited use (demo of GCE launch, we could start over 1000 raw vms and be doing large scale compute in minutes during the live demo (unfortunately urs wasn't confident so we pretaped that part). Ansible is not bad but tf is Terrible. Good idea, terrible execution and evolution

philliphaydon 4y ago

> So, instead of learning Puppet or Terraform, you want to implement one from scratch? I definitely don't want to implement from scratch. I share his sentiments. I keep hearing about Puppet Labs (i think is the right one), but I've been to the site a bunch of times and I end up leaving because I have no idea where to start. So I really donno how I would benefit over just using a few bash scripts or powershell scripts to manage my Linux/Windows servers inn AWS. Many different things are like this. The barrier to entry is high, once you get started it becomes easier and you tend to agree that the tool is much better than other alternatives. However I today learned that Pinia is the state management library that replaces Vuex in Vue. (effectively V5 of Vuex) and I opened up the documentation, and within a few minutes of reading I had it up in running in my little side project. The documentation explained why its now Pinia, and gave a concrete working example of a simple counter and how its used. Vue, Rust, Postgres - these all have my favourite documentation.

randomswede 4y ago
If a few bash scripts is enough for you to manage your servers, you probably don't need puppet (or ansible, or chef). But, yes, I agree that the puppet documentation is (or at least was, I have not looked at it for several years) not entirely brilliant. It is an OK reference manual, but like so much documentation, it is more difficult than necessary for learning the thing.

ratww 4y ago

I don't understand how you were able to make the jump to "I don't like tools that lie about complexity" to "I want to build tools from scratch", but I share GP's complaints. The problem is clearly with tools that promise simplicity for users but are actually only simple for the creators of the tools themselves. For end users there's still a lot of complexity. And it's not that complexity is a problem per se. It's just that it exists on multiple levels. And some of those tools aren't abstracting much or anything at those levels. They're often just providing yet another thin API for something that's still complex for their users, so they haven't really made the hard thing simple, maybe they simplified the 10% of it that don't matter at the user's scale. If anything they introduced even more complexity. Aka, leaky abstractions. Of course, YMMV.