15 ms·
No offense to the creator of YAML, but: The fact that it became one of the de-facto standards for cloud tooling is an absolutely damning statement about the sta
by MenhirMike 3y ago
No offense to the creator of YAML, but: The fact that it became one of the de-facto standards for cloud tooling is an absolutely damning statement about the state of the industry.
I get that XML is about as sexy as mainframes, and that a lot of folks here probably have PTSD from working with Java/Spring web apps, but YAML is about the worst of all worlds.
Though I think the real problem is that real-world configuration files are way too complicated for a simple/dumb/logic-less representation like a .ini/.conf file, so someone thinks to add some logic to is - which is just config-as-code. In a terrible programming language.
If you want config-as-code (and you want to!), just do it properly and use a proper programming language for it. Don't care which one, be it JavaScript, Python, Go, PDP-11 Assembly, or Rust. But please stop with these half-measure DSLs that just don't cut it.
- jandrese 3y agoAlso, config files that are also programs are a classic security vulnerability. Fun!
- richbell 3y agoThe author of SnakeYaml argued vehemently that a library executing untrusted code by default was a feature that people wanted. https://bitbucket.org/snakeyaml/snakeyaml/issues/561/cve-2022-1471-vulnerability-in https://bitbucket.org/snakeyaml/snakeyaml/issues/561/cve-202...
- lodovic 3y agowow, what a read. I'm now convinced to immediately remove that library if I ever come across it.
- ludwik 3y agoWow, this is truly terrifying. It reads as if Donald Trump was the maintainer of a popular library. "Low quality tooling! Sad!!!". He claims with a straight face that every software using his library only loads YAMLs from sources 100% trusted to execute code. He is given example after example to the contrary, which he ignores instead opting to constantly blame "low quality tooling" for generating "false reports" about his perfect software. People can be really weird sometimes.
- sheepshear 3y agoA lot of people misunderstand what he's actually saying. There are two categories of constructors. One is for data that should not be executed, the other is for trusted data that should be executed. There are two libraries. One has default constructors that can execute data, the other has default constructors that don't execute data. He's saying to rtfm and choose the library with the correct defaults, choose the correct constructor from that library, and stop trying to take away the choice.
- richbell 3y ago> He's saying to rtfm and choose the library with the correct defaults, choose the correct constructor from that library, and stop trying to take away the choice. Nobody was trying to "take away the choice". The problem is that you have to explicitly opt-in to be safe. If you followed the code snippets from the README, your application would be vulnerable to RCE without you realizing it; as people pointed out, it would be more secure to have Constructor (safe by default) + DangerousConstructor rather than Constructor (unsafe by default) + SafeConstructor. His argument was that "100% of applications using SnakeYaml do not accept untrusted data".
- sheepshear 3y agoI understood him to be speaking tautologically, that you trust the data when you choose the trusting library without using the safe constructor, whether or not you realize the implication. He seems very well informed that some people are using the trusting constructors on untrusted data. As he explained, this library is, by design, convenient by default. Those seeking safe by default should consider using the other library. "take away the choice" is my summary of several comments that would have the feature removed. One was about how its existence is a vulnerability if file access is compromised. Another was about how code execution is not in the spec. And so on.
- richbell 3y ago> I understood him to be speaking tautologically, that you trust the data when you choose the trusting library without using the safe constructor, whether or not you realize the implication. He was speaking literally; he even rejected several of the provided examples because "users have to login first, therefore the data is trusted" which isn't an argument that any security-conscious person would make. > As he explained, this library is, by design, convenient by default. Those seeking safe by default should consider using the other library. That is a negligent mindset to have. Log4J added the ability to execute arbitrary dns and ldap calls for the sake of convenience, which resulted in one of the most consequential vulnerabilities of the past decade. Opt-in security is dangerous and should never be the default — especially when the feature in question is executing arbitrary input.
- legends2k 3y agoTrue but that's to be tackled differently instead of not providing scripting at all.
- TeMPOraL 3y agoWell, let's be honest: security sucks out all the joy from everything. Even before freedom, the first victim of security is fun.
- jandrese 3y agoOtherwise phrased as "this is why we can't have nice things".
- kevin_thibedeau 3y agoTcl actually excels as a configuration format as it has a simple syntax and you can initially restrict the available commands to a safe, non-Turing subset then add them back in piecemeal as they become necessary for more powerful scriptable setups.
- echelon 3y ago> If you want config-as-code (and you want to!), just do it properly and use a proper programming language for it. Starlark was designed for this purpose.
- packetlost 3y agoConfig as code is basically the default for (Python) Django applications, but for compiled language you would need to bake in a scripting engine or something. There's always sqlite, which is almost everywhere and very fast.
- mkl 3y agoLua is used quite a bit for that purpose, as it's small and easy to embed. I'm not a Lua programmer, but bodging together a few snippets of Lua with a well-defined configuration API is much easier than trying to do that with SQLite statements.
- thadt 3y agoIndeed. About 15 years ago I picked up Lua for exactly that purpose on an embedded device that needed a somewhat sophisticated configuration. Readable. Comments included. If/else for a few things when needed. Was the best configuration experience I've had before or since.
- packetlost 3y agoLua is a fantastic embedded language! I'm partial to Lisps/Schemes for this purpose, EDN/S-Exprs are pretty good config formats as well.
- qudat 3y agoI share the same sentiment. Provide frameworks not config DSLs. People eventually want logic inside their config objects. We can argue how much logic but at a certain point, just using a popular programming language just makes sense because it’s familiar. I’d like my coding expertise translate into this part of the ecosystem.
- wokkel 3y agoPeople want flying cars and free lunches as well. Logic in config shows a defect in your design, not a feature of the system.
- usrusr 3y agoThat's where JSON shines, it's a properly "dead" format that can be trivially opted into allowing logic with the old console.log JSON.stringify, without even touching the unaffected lines (and all the git history they might be associated with). Certainly more trivially in some environments than in others, but I suppose it's close enough to the sweet spot of "adding logic shouldn't be too easy/shouldn't be too hard" wether the project is already nodeish or not. The only downside is that if the objects in question happen to be properly documented in typescript, you will never want to go back. If only there had been a formalized side by side from the start between JSON, the clean serialization format, and the JSON superset for human authoring (comments and quotes anarchy) that people have been informally reinventing hundreds of times...
- AtlasBarfed 3y agoXML does not, in a single line of code with no preknowledge of the document, deserialize into a map or array (of nested maps/arrays as needed). It cannot be mapped easily into domain objects/datastructs without extensive mapping info. Instead, you need to describe the structure of the XML, have preknowledge of prefixes and meanings for namespaces, have to deal with CDATA crap, have directives, config-in-comments, and hosts of other annoyances. XML sucks. I programmed from 1995 to the present. XML sucks. YAML is far far far far far superior. The ONLY good thing about XML is XPath. That's it! XSLT? awful. schemas and other validations? horrid. XStream (java library) was the only thing that made XML usable, and the second JSON (and later YAML) came out, I dropped it immediately.
- lolxmlhateonhn 3y agoThat’s called “schema.” It’s a pretty out there concept, I know, the idea that you should be burdened to document the structure and intent of your data for both human and mechanical consumption. I realize I’m being forceful here, but keep in mind, you are compensated incredibly well. If it takes you another hour to save ten down the road, earn the pay. I don’t understand this aversion to tough stuff - which seems to be pretty popular here - and I’m starting to think I should interview for it a bit harder than I already do. The problem with this thinking is that you, personally, are then forbidden from arguing for the use of a strictly typed language for development because it’s the opposite position to the one you’re holding here. The exact reasons we use languages like those are the same reasons we should be explicit with our schemas. It’s unfortunate that many people try to argue both sides due to the convenience, as you say, of a single line parse, when years of experience has taught that duck anything is a bug fountain. (Not saying you are arguing both, by the way, it’s just common.) Try reading back your gripe with the following in mind: do I have a stronger complaint than “it’s difficult” here? I think you’ll find that you don’t convey one effectively.
- esbeeb 3y agoThere's a happy medium to be found here - balancing ease of use, while avoiding the bug fountain. Having said that, IMHO we should err on the side of avoiding the bug fountain. Make it as simple as possible, but not simpler. Robbing the future with deceptive over-simplicity - by creating a bunch of future difficult debugging scenarios and possibly footguns - is the far worse evil, than missing out on the maximally-convenient onboarding (which can be foolishly optimized for, for the sake of short-term popularity). All such crap-tastic solutions will eventually need to be replaced again anyway, creating an endless, hellish, slow churn.
- ActorNightly 3y agoTo be fair, Amazon is pushing cdk with typescript as the best way to define infrastructure.
- koolba 3y agoLast I checked, CDK was just a typescript veneer atop the steaming pile that is known as CloudFormation. Not even tagged unions can polish that turd.
- Spivak 3y agoAnd it always will be, cdktf is the same but on Terraform objects. At some point you have to reach the end and spit out the final list of stuff you want. If you want full imperative just use the AWS sdk or Ansible but I think people have realized how not maintainable that pattern is.
- emodendroket 3y agoSure, like C is just a veneer on top of machine language.
- baq 3y agoEvery time I need to do something in CDK I am reminded of https://xkcd.com/2347/ https://xkcd.com/2347/ except the whole thing is my config and the tiny bit is the thing I want to understand how to change.
- ActorNightly 3y agoCFN is also a bunch of abstraction over api calls. You can set up infrastructure using boto3 if you want, for free.
- Cloudef 3y ago> If you want config-as-code (and you want to!), just do it properly and use a proper programming language for it. Or generate it, or use something like nix
- kumarvvr 3y agoMost people who lambast XML probably have never used XML or never had to need XML. I designed a complex data acquisition system and after a lot of research, I settled on XML as the only viable option for complex user configuration, that is both readable and rich in content. I then built a UI system that works with the XML and generates cofig docs with ease. Sure, XML has been used in SOAP like systems, and it rightfully gets a bad rap, but that is more on the user than the technology. A hierarchical document with values and attributes and custom tags? That is an almost DSL in itself.
- altcognito 3y agoI can’t help but agree that if we had just built better tools, xml would have had better adoption.
- fzeindl 3y agoI find XML tool in general feature complete and great. Especially editor integration works great as opposed to YAML which is so ambiguous that it IntelliJ IDEA constantly breaks indentation during copy and paste.
- Spivak 3y agoAnd better conceptual docs because it's supposed to be that elements are complete deserializable objects. Having to resort to xpath as the default way of poking at XML where JSON has a good implicit default schema made XML feel so clunky.
- wokkel 3y agoAt least xml has xpath ;-) the implicit Jason to object mapping has failed me on character encoding (no way to specify that in json). Stupid type errors (date/large numbers).
- MenhirMike 3y agoI actually think that XML has some of the best tooling in the entire industry. The problem is: It's mostly commercial tooling, which costs money. Stuff like Oxygen or XMetaL is pretty neat, and I always found Visual Studio's "Create Web Client from SOAP" pretty useful (especially if the Web Service is written in .NET, since it auto-generates a proper WSDL file).
- vbezhenar 3y agoI know that everyone hates YAML, but in my experience it's not that bad. I'm not using it to its full power, I guess, more like safe subset, and it works fine for me. XML has its place, but I wouldn't want to replace my yamls with XML.
- _ZeD_ 3y agoThem why not json?
- vbezhenar 3y agoYAML is way less verbose than JSON. Something as simple as multiline script is terribly suited for JSON.
- baq 3y agoThe problem is when you want to put a script into yaml you should stop and think what it is you’re actually doing.
- EamonnMR 3y agoIf you want something editable by hand, yaml is much nicer than Json because you get comments and you don't need to worry about missing a bracket.
- mistertoolbox 3y agoI have no big love for json or yaml, but i'd gladly take worrying about missing brackets over worrying about the handful of whitespace-related gotchas in yaml.
- munksbeer 3y ago[dead]
- emodendroket 3y agoYAML gets cleaner diffs when you edit it.
- quickthrower2 3y agoAll you need is a good XML editor. It could even make it look like YAML so that it is nice at read time, and even let you edit it like you would a YAML but save it as XML and enjoy the validation etc. tooling.
- appplication 3y agoI’ve felt like Python for config works pretty well, but I understand there’s a whole segment of folks who get twitchy at the thought of Python for anything. But just about every system has it or can install it, and you can do as much config-as-code as you want with it.
- emodendroket 3y agoYAML is easy to read, everything has a parser for it, and it plays well with version control, which cannot be said of most serialization formats. Not sorry for using it in projects and will continue to do so.
- pjmlp 3y agoXML not only everything has a parser for it, it has schema validation and transformation tools across schemas.
- emodendroket 3y agoYes but it is not easy to read (or write) and version control works less well with it than YAML. And having more complex features is part of what people disliked about XML in the first place.
- pjmlp 3y agoHaving sorted out several merge conflicts in Kubernetes configuration spaghetti, I disagree. YAML is only simple on the surface.
- emodendroket 3y agoIf that exact same configuration were JSON or XML would it be easy to sort out? It seems obvious to me that it wouldn’t because the configuration itself is complex.
- pjmlp 3y agoXML definitely, as it is structured, with schema validation.
- emodendroket 3y agoStructure and schema validation are not typically something your VC cares about since it treats everything as text. So I'm not seeing why that resolves the problem.
- gorgoiler 3y agoConfiguration through code requires a lot of organisational discipline. That’s easy to do if it’s just you and your own code. It’s easy if you have good social connections between the people who review each others code (and you also have code review.) One bad apple can ruin everything though: from config import School, Teacher, Course, MRS MNH = Teacher(MRS, “Marissa”, “Neve”, “Harman”) BIO = Course(“Biology”, MNH) ST_SIMONS = School([BIO]) Oh what a nice tidy config you have there! Let’s ruin it! courses = [BIO] if str(today()) < “2023-04”: courses.pop(BIO) if today() > DIVORCE: for c in courses: if c == BIO: c.teacher.name = (“Marissa”, “Cox”) c.teacher.title = “Ms” # gibberish ad nauseam I suppose it’s possible to write nonsense code in any language including YAML, so I don’t know if this is a very good point or not. To put it diplomatically though: your cadre of YAML editors, should they be moved over to using Python, are probably the ones who need the most help writing clear code.
- lodovic 3y agoI'm not convinced that trading a whitespace sensitive DSL for a whitespace sensitive dynamic scripting language will solve the issue.
- TeMPOraL 3y agoI think whitespace sensitivity was not the point of the exercise. But on that topic, I feel whitespace sensitivity is a bad idea even for generic-audience configs. People learn how to use parentheses in primary school. Explicit grouping and nesting isn't black magic.
- Smaug123 3y agoIn general I am sympathetic to whitespace - I find the argument "whitespace insensitivity means the code now contains two conflicting sources of truth: whitespace which the programmer uses, and braces which are the only one that matters because that's what the computer uses" to be compelling. But the whitespace sensitivity of YAML is particularly bad because of e.g. Helm. Templating a whitespace sensitive language with string interpolation is such a terrible idea.
- lelanthran 3y ago> No offense to the creator of YAML, but: The fact that it became one of the de-facto standards for cloud tooling is an absolutely damning statement about the state of the industry. I think the creation and take-up of yaml over something like s-expressions is an indicator that the influential practionors in the cloud industry are all young and inexperienced. If the creators of all these cloud tools display such poor judgment, it makes the tool itself suspect.
- deleted 3y ago[deleted]
- whateveracct 3y agoNix, Dhall, EDN, any extended lambda calculus are all superior to the languages you mention for config as code.
- Aeolun 3y agoI think it’s because XML was just that painful to deal with. When someone offered an alternative, any alternative, that didn’t look utterly crazy, people grabbed it with both hands. All the languages/tools that use it as a config/data format were started during that period, before people realized that all they’d seen of yaml were toy examples.
- esperent 3y agoI think YAML has a particular valid use case: short, well defined, human readable config files that need to be edited by less technical users. The idea is that there's a single config.yml file in the project root. It's mostly empty but could grow to ~100 lines if all config options are are changed, which realistically will be almost never. Most times it'll be 0-20 lines long. In that case xml would be needlessly confusing. It's not easy for non technical users to read.
- wokkel 3y agoNon technical users are usually sufficiently confused by the git tooling to get to said file..I don't see the point.
- oweiler 3y agoWhen programming in Groovy, using Groovy maps as config was pretty common, and allowed you to sprinkle a little bit of logic here and there is things got complicated. But it was just Groovy, no special cases, no weird escaping rules.
- edem 3y agoHah, I wrote my comment before reading the top comment (yours) and I agree. Java PTSD ruined XML, but it is still - to this day - superior to everything else that came after it because of XSD / XSLT and the like.
- chii 3y ago> because of XSD / XSLT and the like. really?! XSLT is an abomination. XSD is bearable since it gives you the validation and autocompletion etc.
- erhaetherth 3y agoEhh, I don't know. Sometimes I want to put code in my config, but there's value in not. It's more portable and easier to parse with various tools and different languages if you stick to some format.
- RGBCube 3y ago> If you want config-as-code (and you want to!), just do it properly and use a proper programming language for it. Don't care which one, be it JavaScript, Python, Go, PDP-11 Assembly, or Rust. But please stop with these half-measure DSLs that just don't cut it. I wish more tooling used Nix, it's a great lisp that isn't ugly.
- Y_Y 3y agoNix, the language? It's very good for derivations (big, composed k:v maps), but it doesn't feel like a lisp, nor would I want to try using it for docker-compose or a config for pre-commit or an omega one description of an ML model parameterisation. Then again I'd love to be shown that this partially-considered opinion is wrong. (Also, isn't Nix's inadequacy as a lisp one of the reasons for the existence of Guix?)
- otabdeveloper4 3y agoNix is JSON + arrow functions + variables. The only difference between Nix and a hypothetical JSON++ is that Nix is lazy by default. I.e., it is nothing at all like any Lisp.
- m45t3r 3y agoI actually use Nix to generate YAML files for GitHub Actions in one of my repositories. It allows me to share code between multiple actions and be consistent with versions/formatting.
- RGBCube 3y agoThat's a really nice use case! Could you provide a link?
- m45t3r 3y agoSorry, didn't see this comment. It is here: https://github.com/thiagokokada/nix-configs https://github.com/thiagokokada/nix-configs In the `actions` directory is all the Nix files. There is some glue code in `lib/flakes` to generate the YAML files from Nix.
- larodi 3y ago> I get that XML is about as sexy as mainframes, and that a lot of folks here probably have PTSD from working with Java/Spring web apps, but YAML is about the worst of all worlds. I really like your sense of humor, much appreciated. Question is whether the PTSD came out of the XML or the Java usage. Also indeed we're eager to see finally some reasonably good Assembly configs, it'd definitely be a blow in the face of Rust magicians. I would vote for Prolog as a config language, though. If memory serves right - some 10 years ago Matt Sergeant of Perl community did a very good talk about this approach, as result of his explorations of this area. Interestingly he has some definite experience with XML also https://www.xml.com/pub/au/22 https://www.xml.com/pub/au/22.
- MrBuddyCasino 3y ago> If you want config-as-code (and you want to!), just do it properly and use a proper programming language for it. This is a mistake every generation has to repeat over and over again until the end of times. So it is written.
- boredtofears 3y ago> absolutely damning statement about the state of the industry what is the state of the industry then?
- otabdeveloper4 3y ago> config-as-code (and you want to!) No, you don't. Any program that generates a config will itself want to be configurable. Now what?
- _8j50 3y agoI'd rather use a stone tablet than freaking XML!! I will never understand your problem with it. JSON or YAML should be enough for 90% use cases. And INI or TOML the rest. Let the XML relic rest in peace please.
- acjohnson55 3y agoThe problem with using a general-purpose programming language for configuration is that you lose the ability to statically interpret it. Maybe one solution is to make sure the configuration SDK is fully side effect free, so that it's always easy to run the configuration with fixed inputs and get a deterministic output.
- raincole 3y ago> The fact that it became one of the de-facto standards for cloud tooling is an absolutely damning statement about the state of the industry. The state of the industry is... perfectly fine? I really don't get why people hate YAML. To me the "programming language as config" idea is just like Lisp-like macro. Yeah it's powerful, but once you have more than 2 people working on it you get DSLs (s for plural). People, at least Lisp people, like to claim the line between code and data is very blurry. In my experience in real world it's blurry 1% of the time. In most cases code is code and data is data.
- PH95VuimJjqBqy 3y agoI've always been of the opinion that YAML is inappropriate as a configuration format, period. It has ambiguous parses, which is an immediate no-no. I remember years ago I was writing my own parser for YAML and when I came across this problem I posted on a few mailing lists and got confirmation that the behavior is per-spec. I've never touched YAML since and still won't. But I miss the days of XML, XSD, and dare I say it ... XSLT. XSLT is stupidly good at what it does as long as you don't abuse it.
- bugeats 3y agoThe Nix Language, while goofy at times, is built for config-as-code and is hiding a decent little functional language in what looks like just attribute assignments. Likewise, CUE Lang is built for config (esp merging docs with shared refs) and is highly under-appreciated. You can express powerful computations if you puzzle over the logical inferencing for a bit. https://nixos.org/manual/nix/unstable/language/index.html https://nixos.org/manual/nix/unstable/language/index.html https://cuelang.org/docs/concepts/logic/ https://cuelang.org/docs/concepts/logic/