3 ms·
Reminds me of a fatal flaw of yaml. Turns out truncating a yaml file doesn't make it invalid. Which can lead to some rather non-obvious failures.
by taeric 2y ago
Reminds me of a fatal flaw of yaml. Turns out truncating a yaml file doesn't make it invalid. Which can lead to some rather non-obvious failures.
- nextts 2y agoWhat is the failure mode where a yaml file gets truncated? They are normally config files in Git. Or uploaded to S3 or Kubernetes etc. CSV has the same failure mode. As does HTML. (But not XML)
- bobmcnamara 2y agoBad merges.
- taeric 2y agoI couldn't find the story on it, but there was an instance of a config for some major service getting truncated, but since it was yaml it was more difficult to figure out that that was what happened. I think in AWS, but I can't find the story, so can't really remember. And fully fair that you can have similar issues in other formats. I think the complaint here was that it was a bit harder, specifically because it did not trip up any of the loading code. With a big lesson learned that configs should probably either go pascal string style, where they have an expected number of items as the first part of the data, or xml style, where they have a closing tag. Really, it is always amusing to find how many of the annoying parts of XML turned out to be somewhat more well thought out than people want to admit.
- pasc1878 2y agoSame is true of CSV/TSV.
- taeric 2y agoI think you are a bit more likely to notice in a CSV/TSV, as it is unlikely to truncate at a newline? Still, fair point. And is part of why I said it is a flaw, not the flaw. Plenty of other reasons to not like YAML, to me. :D
- pasc1878 2y agoNot if it is split at a line e.g. if the source or target can only deal with a fixed number of lines.
- taeric 2y agoRight, that is what I meant about that being unlikely? Most instances of truncated files that I have seen were because of size, not lines. Still, a fair point.
- ziml77 2y agoReally depends on how the CSV is generated/transferred. If the output of the faulting software is line-buffered then it's quite likely that a failure would terminate the file at a line break.