3 ms·

I agree with most of his points, but disagree on some: - The value of a CI/CD pipeline is the value of its output. Doesn't matter if it took 5 seconds or 5 day

I agree with most of his points, but disagree on some: - The value of a CI/CD pipeline is the value of its output. Doesn't matter if it took 5 seconds or 5 days. I have worked on pipelines that were shitty and long but delivered enormous value, and quick ones that were noisy and useless. - The "code is better than yaml" post misunderstands what YAML is, and then says you should use a general purpose programming language for configuration. But if you're programming, you're not configuring, you're programming. My point is, both code and YAML are completely wrong solutions to the problem. Look at configuration file formats for 20+ year old programs as an example of good configuration. (Hint: none of them are either YAML or code) - "Release early, Release often" only works for certain products/services. It's not a good idea to deploy 10x a day to an airplane in flight. (No, contrarian who's about to reply, it's not a good idea, just shut up.) - Declarative configuration isn't a thing. It's just configuration. Non-declarative configuration isn't a thing either, that's just instructions. Declarative doesn't mean anything. It's a red herring people repeat because they heard someone describe it and it sounded smart. - Do not 'set -o pipefail' by default. It will only waste your time when your scripts start failing because some command in the middle of a pipe returned non-zero while still outputting the result you wanted, and you spend 3 hours adding debugging to figure out where the error was coming from and write some extra error handling code. Just check if the result of the pipe looks accurate. Same result, fewer failures. - If you have to do a simple task more than 3 times, check if (T*N)>A, or if it's a value chain bottleneck. If neither is true, don't automate yet.

lazyant 4y ago
> My point is, both code and YAML are completely wrong solutions to the problem. Look at configuration file formats for 20+ year old programs as an example of good configuration. (Hint: none of them are either YAML or code) I take configuration as custom values for an environment, read at start time by the executable code, that are easily readable for humans (a non-programmer ops person could change) and easy to produce/read/manipulate. From that point of view, coding is different as you say, while it gives you more power it's just not very readable. All the config files I've worked on for 20+ years are a way of expressing KEY=value, with some nesting allowed to not repeat yourself (.ini, Yaml, json, even XML basically) so in my mind they are all basically equivalent.
- 0xbadcafebee 4y ago
  Some examples of config files that are not KEY=VALUE: Apache, X11, resolv.conf, sudoers, syslog.conf, mtab, inittab/gettytab, ntp.conf, passwd/group/shadow, ldap.conf, PAM, openssh, CUPS... just a few on my desktop. These files support a wide variety of functionality beyond what a data-serialization format like YAML support. The functionality is designed for humans to input data germain to the configuration of the application in an easy way. That is to say, it both makes the admin's job easier, and tells the program what it wants to know. Data serialization formats (JSON, YAML) are particularly poorly suited for configuration. JSON doesn't even support comments. YAML does, but then forces you to correctly format the document in a non-human-intuitive way, making the admin's job harder, and not necessarily telling the program what it wants to know. It has been well established how many problems humans encounter editing YAML files, to the point that people just take out features to make it less problematic, though of course some problems remain. They could eliminate these problems by using an established robust configuration format, or even making up their own, but modern programmers have a poor education in software design and avoid anything that isn't trendy or popular. .INI is a decent enough format, with the caveat that it was never a standard, so parsing is always application-specific. At that point you might as well just make your own configuration format that serves the admin and programmer better than trying to force complex configuration into simple KEY=VALUE pairs. XML is sometimes actually useful, but time has shown that it is not human friendly, and often not even program friendly.
nikau 4y ago
> - Do not 'set -o pipefail' by default. It will only waste your time when your scripts start failing because some command in the middle of a pipe returned non-zero while still outputting the result you wanted, and you spend 3 hours adding debugging to figure out where the error was coming from and write some extra error handling code. Just check if the result of the pipe looks accurate. Same result, fewer failures. Exactly how do you "check if the result of the pipe looks accurate" without having to get a human to check the log file? The whole point of automation is to get rid of human interaction even if it requires some upfront work to implement error handling where appropriate...
- 0xbadcafebee 4y ago
  Well it's context-dependent, but there are many different strategies. Most well designed command-line programs output to STDERR if they have an error, and STDOUT if they don't. So the simplest possible method is just to see if STDOUT was 0-length or not. If it's not 0-length, then assume the command completed successfully. grep can help validate the output. Depending on the command, it may resist partial failures; for example, many commands that output a JSON blob will wait until they have all their data before outputting the blob. If the script you're writing depends on always working 100%, then you can do a lot more work to check for errors. First you would enable set +e, because if you exit on error with pipefail, you can't even check what the error was from PIPE_STATUS. Instead you can inspect every pipe result and tell your user what part failed, and decide if you should continue, print only potentially partial results, retry the operation, or fail immediately. Most programmers don't handle error conditions properly and just immediately fail on any error, which itself causes errors which needs a human to inspect a log. Worst case, the shell script without pipefail just keeps working, when with pipefail it would have failed a lot.

lazyant 4y ago

> My point is, both code and YAML are completely wrong solutions to the problem. Look at configuration file formats for 20+ year old programs as an example of good configuration. (Hint: none of them are either YAML or code) I take configuration as custom values for an environment, read at start time by the executable code, that are easily readable for humans (a non-programmer ops person could change) and easy to produce/read/manipulate. From that point of view, coding is different as you say, while it gives you more power it's just not very readable. All the config files I've worked on for 20+ years are a way of expressing KEY=value, with some nesting allowed to not repeat yourself (.ini, Yaml, json, even XML basically) so in my mind they are all basically equivalent.

0xbadcafebee 4y ago
Some examples of config files that are not KEY=VALUE: Apache, X11, resolv.conf, sudoers, syslog.conf, mtab, inittab/gettytab, ntp.conf, passwd/group/shadow, ldap.conf, PAM, openssh, CUPS... just a few on my desktop. These files support a wide variety of functionality beyond what a data-serialization format like YAML support. The functionality is designed for humans to input data germain to the configuration of the application in an easy way. That is to say, it both makes the admin's job easier, and tells the program what it wants to know. Data serialization formats (JSON, YAML) are particularly poorly suited for configuration. JSON doesn't even support comments. YAML does, but then forces you to correctly format the document in a non-human-intuitive way, making the admin's job harder, and not necessarily telling the program what it wants to know. It has been well established how many problems humans encounter editing YAML files, to the point that people just take out features to make it less problematic, though of course some problems remain. They could eliminate these problems by using an established robust configuration format, or even making up their own, but modern programmers have a poor education in software design and avoid anything that isn't trendy or popular. .INI is a decent enough format, with the caveat that it was never a standard, so parsing is always application-specific. At that point you might as well just make your own configuration format that serves the admin and programmer better than trying to force complex configuration into simple KEY=VALUE pairs. XML is sometimes actually useful, but time has shown that it is not human friendly, and often not even program friendly.

nikau 4y ago

> - Do not 'set -o pipefail' by default. It will only waste your time when your scripts start failing because some command in the middle of a pipe returned non-zero while still outputting the result you wanted, and you spend 3 hours adding debugging to figure out where the error was coming from and write some extra error handling code. Just check if the result of the pipe looks accurate. Same result, fewer failures. Exactly how do you "check if the result of the pipe looks accurate" without having to get a human to check the log file? The whole point of automation is to get rid of human interaction even if it requires some upfront work to implement error handling where appropriate...

0xbadcafebee 4y ago
Well it's context-dependent, but there are many different strategies. Most well designed command-line programs output to STDERR if they have an error, and STDOUT if they don't. So the simplest possible method is just to see if STDOUT was 0-length or not. If it's not 0-length, then assume the command completed successfully. grep can help validate the output. Depending on the command, it may resist partial failures; for example, many commands that output a JSON blob will wait until they have all their data before outputting the blob. If the script you're writing depends on always working 100%, then you can do a lot more work to check for errors. First you would enable set +e, because if you exit on error with pipefail, you can't even check what the error was from PIPE_STATUS. Instead you can inspect every pipe result and tell your user what part failed, and decide if you should continue, print only potentially partial results, retry the operation, or fail immediately. Most programmers don't handle error conditions properly and just immediately fail on any error, which itself causes errors which needs a human to inspect a log. Worst case, the shell script without pipefail just keeps working, when with pipefail it would have failed a lot.