11 ms·
YAML: The Norway Problem (2022)
- firesteelrain 2y agoThis problem occurs because pyyaml load() uses the full YAML 1.1 schema. There is another function BaseLoader that will interpret everything as a string which is the workaround that the article suggests. Just another way to achieve it. It’s a bit of a sore spot in the YAML community as to why PyYAML can’t / won’t support YAML 1.2. It was in maintenance mode for a while. YAML 1.2 also introduced breaking changes. From a SO comment: “ As long as you're okay with the YAML 1.1 standard, PyYAML is still perfectly fine, secure, etc. If you want to support the YAML 1.2 spec (released in 2009), you can use ruamel.yaml, which started out as a fork of PyYAML. – CrazyChucky Commented Mar 26, 2023 at 20:51” - https://stackoverflow.com/q/75850232 https://stackoverflow.com/q/75850232
- rat87 2y agoYeah it's a problem I had to put up a PR on a tool I was using because I ran into the Norway problem on yaml I was getting from another team. I did ask them to add quotes just in case
- firesteelrain 2y agoA supplier we contracted with and we gave requirements to asked me what format do we want the export/import of the data to be in and I said JSON. It’s simple, easy and can be converted into anything else very easily
- gschizas 2y agoI wish that ruamel.yaml had better documentation. I've had to dive into the code so many times to find out how to do something.
- gnabgib 2y agoRelated The YAML document from hell (566 points, 2023, 353 comments) https://news.ycombinator.com/item?id=34351503 https://news.ycombinator.com/item?id=34351503 That's a Lot of YAML (429 points, 2023, 478 comments) https://news.ycombinator.com/item?id=37687060 https://news.ycombinator.com/item?id=37687060 No YAML (Same as above) (152 points, 2021, 149 comments) https://news.ycombinator.com/item?id=29019361 https://news.ycombinator.com/item?id=29019361
- mdaniel 2y agoAnd some light commentary a few days ago: https://news.ycombinator.com/item?id=43648263 https://news.ycombinator.com/item?id=43648263 - Apr 2025 (51 comments)
- codr7 2y ago[flagged]
- dissent 2y agoI reckon if this is really a big concern for anybody, then they are probably writing way too much YAML to begin with. If you're being caught out by things like this and need to debug it, then it maps very cleanly to types in most high level languages and you can generate your YAML from that instead.
- makeitdouble 2y agoSadly you usually realize you've been writing too much YAML way past the turning point, and it will be a pain to move a single file to JSON for instance when you have a whole process and system that otherwise ingest YAML, including keeping track of why this specific part of JSON and not YAML. So people work around the little paper cuts, while still hitting the traps from time to time as they forget them. > generate YAML I've a hard time finding a situation where I'd want to do that. Usually YAML is chosen for human readability, but here we're already in a higher level language first. JSON sounds a more appropriate target most of the time ?
- charrondev 2y agoIsn’t yaml a strict superset of JSON? Any compliant YAML parser should be able to ingest a JSON document.
- mannykannot 2y agoAre there no cases where well-formed JSON could be subject to the problems covered in the article, when parsed by a compliant YAML parser? I'm asking because I know nothing about YAML and not much more about JSON.
- charrondev 2y agoNot that I know. JSON requires strings to be quoted which is basically the problem here. Of course it’s not a great human writable configuration format (no comments being a huge problem). I’m just pointing out that it should be very simple to swap a YAML file for a JSON file in any system that accepts YAML
- ashishb 2y agoHow often do people even encounter this issue? I have been using YAML for 5+ years and have never had it before. Further, I use `yamllint` which points this out as a lint issue "truthy value should be one of [false, true]".
- rat87 2y agoI have when getting an openapi yaml file from someone else.
- mongol 2y agoHas been encountered where I work. A global website with lots of country-specific config.
- hinkley 2y agoFractions are discriminatory when they happen to one individual or group every time or even just the first time. See also p95 but the same couple of users always see the p99 time, due to some bug.
- ashishb 2y agoIndeed, based on the comments, it is a scissor-bug. Most people never encountered it while some encountered it a lot.
- Y-bar 2y agoNever experienced it for the past 10+ years since the bug was fixed in the spec.
- speedgoose 2y agoI have encountered it once, though I live in Norway and worked in IT there for a decade.
- tetha 2y agoI don't recall encountering the norway problem in the wild. Ansible has a pretty common issue with file permissions, because pretty much every numeric representation of a file mode is a valid number in YAML - and most of them are not what you want. Sure, we can open up a whole 'nother can of worms if we should be programming infrastructure provisioning in YAML, but it's what we have. Chef with Ruby had much more severe issues once people started to abuse it. Plus, ansible-lint flags that reliably.
- thyrsus 2y agoI do a lot of ansible which needs to run on multiple versions, and their yaml typing are not consistent - whenever I have a variable in a logic statement, I nearly always need to apply the "| bool" filter.
- polski-g 2y agoYep. I just want strict yaml: anything encased in quotes is a string, anything not is not a string (bool, int or float)
- deleted 2y ago[deleted]
- awestroke 2y agoIncluding keys?
- mdaniel 2y agoThis is likely hair splitting, but you are far more likely getting bitten by the monster amount of variance in jinja2 versions/behaviors than by anything "yaml-y" For example, yaml does not care about this whatsoever - name: skip on Tuesdays when: ansible_date_time.weekday != "Tuesday" but different ansible versions are pretty yolo about whether one needs to additionally wrap those fields in jinja2 mustaches - name: skip on Tuesdays when: '{{ ansible_date_time.weekday != "Tuesday" }}' and another common bug is when the user tries to pass in a boolean via "-e" because those are coerced into string key-value pairs as in $ ansible -e not_today=true -m debug -a var=not_today all localhost | SUCCESS => { "not_today": "true" } but if one uses the jinja/python compatible flavor, it does the thing $ ansible -e not_today=True -m debug -a var=not_today all localhost | SUCCESS => { "not_today": true } It may be more work than you care for, since sprinkling rampant |bool likely doesn't actively hurt anything, but the |type_debug filter[1] can help if it's behaving mysteriously 1: https://docs.ansible.com/ansible/11/collections/ansible/builtin/type_debug_filter.html#ansible-collections-ansible-builtin-type-debug-filter https://docs.ansible.com/ansible/11/collections/ansible/buil...
- quechimba 2y agoWe had this issue many years ago when people from Norway couldn't sign up. Took us a while to figure out
- duxup 2y agoOr were they from Noway ...
- deleted 2y ago[deleted]
- magicalhippo 2y agoAs a Norwegian I'm very curious, where in the pipeline were you using YAML? And why? I've only seen it used for configuration.
- StableAlkyne 2y agoI've seen teams use it as a replacement for JSON because it has the perception of being more "modern"
- deleted 2y ago[deleted]
- bornfreddy 2y agoWhile JSON is annoying because it lacks some pretty basic features (comments, trailing comma), at least its spec is short. YAML is huuuge - there are way too many ways to do the same thing.
- tough 2y agousually locale's paths gone wrong
- quechimba 2y agoWe were using a fork of https://github.com/carmen-ruby/carmen/tree/master/iso_data/base/world https://github.com/carmen-ruby/carmen/tree/master/iso_data/b... with our own translations. We used the data in the signup form.
- singpolyma3 2y agoQuote your strings
- pavel_lishin 2y agoEmpty the footgun before firing.
- School-Cotton 2y ago“Be liberal in what you accept” rears its ugly head once more.
- eyelidlessness 2y agoBeing liberal in what you accept, also known as the “robustness principle”, doesn’t mean being ambiguous or surprising about how you accept it. If anything, robustness requires a great deal more precision and clarity (at least with your own reasoning, then with how you communicate what to expect from it).
- kazinator 2y agoPostel's Law does not deserve to be nicknamed the Robustness Principle. Robustness has a meaning and it refers to handling bad inputs gracefully. An example of a lack of robustness is allowing a malicious actor to execute arbitrary code by supplying a datum larger than some buffer limit. Trying to make sense of invalid inputs and do something with them isn't robustness. It's just example of making an extension to a spec. The extension could be robust or not. Postel's Law amounts to "have extensions and hacks to handle incorrectly formatted data, rather than rejecting them. So, OK, yes, that entails being robust to certain bad inputs that are outside of the spec, but which land into onto one of the extensions. It doesn't entail being robust to inputs that fall outside of the core spec and all hacks/extensions. Cherry picking certain bad inputs and giving them a meaning isn't, by itself, bona fide robustness; robustness means handling all bad inputs without crashing or allowing security to be compromised.
- hinkley 2y agoPandering to customers will make you a lot of money today but very narrow margins tomorrow. If you’re in startup mentality your bosses may be 100% fine with that. But you will likely be stuck supporting that crap because you didn’t become wealthy in the IPO/merger.
- munch117 2y agoPostel's law isn't about accepting arbitrary invalid inputs. It's about inputs that are technically invalid but the intent is obvious from looking at it, and handling those according to intent. In a distributed non-adversarial setting, this is exactly what you want for robustness. The problem, as we've come to realise in the time since Postel's law was formulated, is that there is no such thing as a distributed non-adversarial setting. So I get what you're saying. But your definition of robustness is too narrow as well. There's more to robustness than security. When Outlook strips out a certificate from an email for alleged security reasons, then that's not robustness, that's the opposite, brokenness: You had one job, to deliver an attachment from A to B, and you failed. Robustness and security can be at odds. It's quite OK to say, "on so and so occasion I choose to make the system not robust, because the robust solution would not be sufficiently secure".
- kazinator 2y agoIn Lisp, if you want to read text into symbols (e.g. file of words), you just switch to a dedicated package in which those symbols are interned. Then if NIL happens to come up, it will be a symbol named "NIL" in that package, unrelated to the special object.
- TZubiri 2y agoThat edge case sounds like a reasonable tradeoff you would make for such a simple and readable generic data format. Escaped json probably hits that sweetspot by being a bit uglier than yaml, but 100 times simpler than xml, though.
- tetha 2y agoMh, since I just commented about ansible, you just made XML-based ansible flash in front of my eyes. I think I'm in a bit of pain now. <tasks> <ansible.builtin.copy notify="restart minio"> <src> files/minio.service </src> <dest> /etc/systemd/system/minio.service </dest> <owner> root </owner> <group> root </group> <mode> 0x644 </mode> </ansible.builtin.copy> </tasks> But you could use XSLT to generate documentation in XHTML from your playbooks about what files are deployed, what services are managed...
- mdaniel 2y agoI will die on the hill than writing out ansible.builtin. are characters of my life I'll never get back, and refuse to. If it's built in why do I have to qualify it?! Also, watch out: 0x644 != 0644 which is the mode you meant
- thund 2y agoI like using tags and avoid any doubt !!boolean https://dev.to/kalkwst/a-gentle-introduction-to-the-yaml-format-bi6#:~:text=A%20YAML%20tag%20is%20a,)%2C%20followed%20by%20a%20URI. https://dev.to/kalkwst/a-gentle-introduction-to-the-yaml-for...
- nnurmanov 2y agoAnother solution is to change the country name:)
- gunalx 2y agoNo way.
- gunalx 2y agoThoug we have renamed amino acidsvi think it was. Because microsoft excel switched the original names to months.
- madcaptenor 2y agoGenes, not amino acids. https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates https://www.theverge.com/2020/8/6/21355674/human-genes-renam...
- gunalx 2y agoThanks for corrections.
- riffraff 2y agoUsual reminder that this is not a problem in YAML 1.2 released 15 years ago. Sadly many libraries still don't support it.
- lifthrasiir 2y agoThis effectively means that a new version of specification didn't solve the problem at all.
- normie3000 2y agoGoogle App Engine used to do this to environment variables defined in YAML. IIRC it would convert the string "true" to "Yes", which was a fun surprise when deploying Java And NodeJS apps.
- stephenr 2y agoIt's not a coincidence that YAML is a perfect acronym for "yet another migraine looming". I mean ok it is technically a coincidence but it definitely feels like the direct result of the "what could possibly go wrong" approach the spec writers apparently took
- raffraffraff 2y agoWhy not just use quotes all the time for strings?
- mystifyingpoi 2y agoI like that in concept, but 1) literally no one does that (prime example - Kubernetes docs) and 2) it looks much more messy with quotes, when you know that they are unnecessary in 95% of cases.
- zelphirkalt 2y agoOh, I did that in Ansible stuff. Using quotes for all strings. Exactly because I know what a mess YAML is.
- mystifyingpoi 2y agoFair enough. I'm tempted to try it. There should be probably a yamllint rule for this...
- kinow 2y agoI guess sometikes it is out of your control. I work on a workflow manager where users specify their workflows with YAML. So there's little we can do to prevent them from writing things like no, n, t in a place it could cause some issue like ij the article.
- zelphirkalt 2y agoAh, the many places that choose to use YAML for no good reason...
- kinow 2y agoYeah, can't say much about it as I joined after they had already decided on using YAML.
- 2y ago
- praptak 2y agoRelated: the YAML exponent problem[0] TLDR: unquoted hex hash in YAML is fine until it happens to match \d+E\d+ when it gets interpreted as a float in scientific notation. [0]https://www.brautaset.org/posts/yaml-exponent-problem.html https://www.brautaset.org/posts/yaml-exponent-problem.html
- ajuc 2y agoYAML is just doing too much and trying to be too clever.
- xelxebar 2y agoThis has been fixed since 2009 with YAML 1.2. The problem is that everyone uses libyaml (_e.g._ PyYAML _etc._) which is stuck on 1.1 for reasons. The 1.2 spec just treats all scalar types as opaque strings, along with a configurable mechanism[0] for auto-converting non-quoted scalars if you so please. As such, I really don't quite grok why upstream libraries haven't moved to YAML 1.2. Would love to hear details from anyone with more info. [0]:https://yaml.org/spec/1.2.2/#chapter-10-recommended-schemas https://yaml.org/spec/1.2.2/#chapter-10-recommended-schemas
- xigoi 2y agoI’m sad that the “fix” was to disallow “no” as a more readable alternative to “false”, rather than to disallow unquoted strings.
- xelxebar 2y agoThe fix is to make conversion user-controllable. If you want to disallow bare scalars except for booleans and numbers or whatever, it's just a little bit of configuration away.
- mckn1ght 2y agoIt’s silly to have so many keyword synonyms as specified in that earlier regex. I’m also glad we can’t specify numeric literals as roman numerals. KISS
- xigoi 2y agoHonestly I’d prefer if “yes” and “no” were the only ways to spell the boolean values. They make sense in pretty much all contexts where booleans are used, whereas “true” and “false” rarely make sense.
- tacker2000 2y agoIn boolean logic true/false is ubiquitious and well known. As you can see, if one tries to be cute with it, one will get all sorts of issues. So at this point it doesnt make sense to use anything else.
- anvandare 2y ago"The limits of my keyboard mean the limits of my programming language." If only they had had ⊥ and ⊤ somewhere on their keys to work with Booleans directly while designing the languages. In another branch of history, perchance.[1] [1] https://en.wikipedia.org/wiki/APL_(programming_language)#/media/File:APL-keybd2.svg https://en.wikipedia.org/wiki/APL_(programming_language)#/me...
- tossandthrow 2y ago⊥ and ⊤ is not entirely congruent to false and true. Boolean and propositional logic is not the same.
- Q6T46nT668w6i3m 2y agoFor _ordinary_ two‑valued classical propositional logic, e.g., YAML, they are congruent.
- rusk 2y agoI have an emacs macro for this
- hgomersall 2y agoIMO the proposed solution of StrictYAML + schema is the right one here and what we use extensively for human readable configs. StrictYAML (linked to in the post) is essentially a string-type-only restriction of YAML, so you impose your type coercion on the parsed data structure.
- vander_elst 2y agoIf you have a schema, why not using directly something like protobufs?
- LelouBil 2y agoThe comment you replied to talks about human readable configs
- Mesopropithecus 2y agoLike this? https://protobuf.dev/reference/protobuf/textformat-spec/ https://protobuf.dev/reference/protobuf/textformat-spec/
- mdaniel 2y agohttps://protobuf.dev/reference/protobuf/textformat-spec/#:~:text=this%20is%20a%20draft%20spec%20reverse-engineered%20from%20the%20c%2B%2B%20text%20format https://protobuf.dev/reference/protobuf/textformat-spec/#:~:... and, setting that aside, the very next paragraph says that this is a legit representation of -2.0 which means something has gone gravely wrong value: - # change this to 3.14 one day 2.0
- weinzierl 2y agoPerl has a Poland Problem. The customary file extension for Perl files is *.pl. This worked well until Apache introduced content negotiation and the convention to add a language code as file extension. It had index.html.en, index.html.de, for example. index.html.pl is where the problem started and the reason why the officially recommended file extension for Perl files used to be (still is?) *.plx. I don't have the Camel book at hand, but Randal Schwartz's Learning Perl 5th edition says: "Perl doesn't require any special kind of filename or extension, and it's better not to use an extension at all. But some systems may require an extension like plx (meaning PerL eXecutable); see your system's release notes for more information."
- dtech 2y agoThat sounds more like an Apache problem than a Perl problem. It's their mistake and it's not even relevant outside Apache context
- weinzierl 2y agoIt should have been an Apache problem, yes. Not only did it turn out that at least the language negotiation part of content negotiation wasn't the best idea but the way Apache handled it was problematic apart from the pl problem. In the end the Perl community took the issue upon them, so historically I'd say it was a Perl problem (of choice).
- maxloh 2y agoThat should be marked as a breaking change on Apache side IMO. It would be a security nightmare if server code were leaked to public.
- ginko 2y agoAlso, Prolog has the Perl problem. :)
- weinzierl 2y agoYes, I think Prolog had pl even before Perl came around, but Perl snatched it away,
- whacko_quacko 2y agoPandas has a Nigeria problem, where NA -> NaN. It's not that bad, because you can explicitly turn that behavior off, but ask me how I know =(
- trueismywork 2y agoHow?
- whacko_quacko 2y agoFor example when reading a CSV. Try using `read_csv` on a file that contains two letter country codes including NA.
- orangewindies 2y agoThat's a Namibia problem, Nigeria is NG.
- whacko_quacko 2y agoDamn, you're right ^^' Thanks for pointing that out
- pkkm 2y agoProgramming with string templates, in a highly complex and footgun-rich markup language, is one of the things I find most offputting about the DevOps ecosystem.
- sph 2y agoI believe Satan itself decided to mix YAML, Jinja and Turing-complete logic when it created Ansible. It truly is the sendmail of the modern era.
- senderista 2y agoSeveral years ago when I was writing a deployment system for a cloud distributed database, I tried to automate everything with Ansible playbooks and the Ansible "API" (LOL). I pretty quickly gave up on implementing anything but the most trivial logic in templated YAML and switched to Python (wrapping maximally-dumb Ansible playbooks) for everything nontrivial.
- Fizzadar 2y agoYou might like pyinfra.
- mdaniel 2y agoJust about every time someone complains about ansible, there's a comment to plug this project but pyinfra seems to opt-out of the cloud provisioning part, instead delegating to its terraform connector, which drags in all the nonsense that entails. That makes it not only less useful but (IMHO) a horrible name for a project that only does "remote execution" and not infrastructure. The fact that it's even missing @aws @azure @gcp connectors further solidifies "who is the audience for this thing?"
- sph 2y agoNot everyone runs cloud servers. pyinfra seems to fit my needs like a glove, so I guess I am the intended audience. I never liked the provisioning overlap Ansible has with Terraform, so it makes sense to me: provisioning servers with tf, configure them with another tool, whether it’s ansible or pyinfra. Well, at least in theory
- azernik 2y agoEven worse is the all-decimal MAC problem. Some genius decided that, to make time input convenient, YAML would parse HH:MM:SS as SS + 60×MM + 60×60×HH. So you could enter 1:23:45 and it would give you the correct number of seconds in 1 hour, 23 minutes, and 45 seconds. They neglected to put a maximum on the number of such sexagesimal places, so if you put, say, six numbers separated by colons like this, it would be parsed as a very large integer. Imagine my surprise when, while working at a networking company, we had some devices which failed to configure their MAC addresses in YAML! After this YAML config file had been working for literal years! (I believe this was via netplan? It's been like a decade, I don't remember.) Turns out, if an unquoted MAC address had even a single non-decimal hex digit, it would do what we expected (parse as a string). This is not only by FAR the more common case, but also we had an A in our vendor prefix, so we never ran into this "feature" during initial development. Then one day we ran out of MAC addresses and got a new vendor prefix. This time it didn't have any letters in it. Hilarity ensued. (This behavior has thankfully been removed in more recent YAML standards.)
- maelito 2y agoWe should use very basic yaml parsers without these kind of functions.
- ghuntley 2y agoSee also https://noyaml.com https://noyaml.com (feel send in PRs with your gripes/gotchas re: YAML)
- alkonaut 2y agoAlways quote all yaml strings. If you have a yaml file that has something that isn't a simple value (number, boolean) such as for example a date, time, ip-address, mac address, country code, phone number, server name, configuration name, etc. etc. then you are asking for trouble. Just DON'T DO THAT. It's pretty simple. "Yeah but it's so convenient" "Yeah but the benefit of yaml is that you don't need quotes everywhere so that it's more human readable" DON'T
- ohgr 2y agoYeah that. 00,01,02,03,04,05,06,07,OH SHIT
- cirwin 2y agoI’ve been working on https://conl.dev https://conl.dev, which fixes/removes YAMLs problematic features. Trying to find a tag-line for it I like, maybe “markdown for config”?
- yimby2001 2y ago[dead]
- endofreach 2y agoThe article mentioned people with the last name "null". I never thought about that. It sounds like really fun in modern days to have that last name.
- mdaniel 2y agoThere have been several write-ups about it: https://news.ycombinator.com/item?id=43113997 https://news.ycombinator.com/item?id=43113997 https://news.ycombinator.com/item?id=15046223 https://news.ycombinator.com/item?id=15046223