3 ms·

I really want to agree because it seems to make so much sense in theory. It gets rid of the need to parse by standardizing on syntax. But it doesn't solve every

I really want to agree because it seems to make so much sense in theory. It gets rid of the need to parse by standardizing on syntax. But it doesn't solve everything, so we still don't get to live the dream. Namely, it does not solve the issue of schemas, versioning, and migration. And this is a really big issue that threatens to derail the whole project. If my script runs a pipeline `a | b | c` and utility b gets updated with a breaking change to the schema, it breaks the entire script. Now I've got to go deep into the weeds to figure out what happened, and the breakage might not be visible in the human-readable output. So to debug I'll have to pass the flag to each tool to get it to print all the json to stdout, and then sit there eyeballing the json to figure out how the schema changed and what I need to do to fix my script. Seems like a big mess to me. Unless there's something I'm missing?

kbrazil 3y ago
There could be a major schema change that breaks the contract, but one of the nice things about JSON output is that it allows the creation of new fields without affecting downstream consumers. That is, if I have a CLI program that spits out a list of IP addresses and one day I want to also output the corresponding dns names, I can simply add the "dns" field and existing pipelines will ignore the field and work just fine. This is better than grep/awking/etc. unstructured text to STDOUT because, depending on how the author decides to add the new field, it can easily break existing pipelines that rely on the shape of the data to stay the same.
mistercow 3y ago
> So to debug I'll have to pass the flag to each tool to get it to print all the json to stdout I'm not sure how that's a particular burden. If you have `a | b | c` and you want to debug the output of b, you already have to pull that out and debug it or `tee` it, because its stdout is being piped otherwise. This would be the same, except that you'd pipe (or tee) "machine out" to stdout. Or, since we're making wild backwards incompatible changes anyway, add a directive to the shell that makes it dump all "machine out" to stdout, a la "set +x". Now you don't even have to change your code. Just wrap the line in `set -o dump-machine-out` and `set +o dump-machine-out`

kbrazil 3y ago

There could be a major schema change that breaks the contract, but one of the nice things about JSON output is that it allows the creation of new fields without affecting downstream consumers. That is, if I have a CLI program that spits out a list of IP addresses and one day I want to also output the corresponding dns names, I can simply add the "dns" field and existing pipelines will ignore the field and work just fine. This is better than grep/awking/etc. unstructured text to STDOUT because, depending on how the author decides to add the new field, it can easily break existing pipelines that rely on the shape of the data to stay the same.

mistercow 3y ago

> So to debug I'll have to pass the flag to each tool to get it to print all the json to stdout I'm not sure how that's a particular burden. If you have `a | b | c` and you want to debug the output of b, you already have to pull that out and debug it or `tee` it, because its stdout is being piped otherwise. This would be the same, except that you'd pipe (or tee) "machine out" to stdout. Or, since we're making wild backwards incompatible changes anyway, add a directive to the shell that makes it dump all "machine out" to stdout, a la "set +x". Now you don't even have to change your code. Just wrap the line in `set -o dump-machine-out` and `set +o dump-machine-out`