4 ms·

While I think that ASCII including a specific record delimiter character (0x1E) that isn’t used is a missed opportunity, I actually think the lack of a set inte

While I think that ASCII including a specific record delimiter character (0x1E) that isn’t used is a missed opportunity, I actually think the lack of a set interchange format is a benefit. No format will work for everyone and if there was one set format, we’d still have people complaining about how it doesn’t work for their usecase. We'd also have had to (finally) answer the question of how to send the schema with the data. JSON (and XML and YAML and…) are all methods to try and adapt a text based data interchange format. And they all work roughly the same. You ultimately need some kind of processing library to full read them. But working with the data in any sort of structured context requires specialized tools (ex: jq). Or you can still grep it, just like I do to manage my tab delimited data, and get part of the way. I think the chaos of having no standard is kinda the point. Text is the lowest format that is easy for people to work with. We can visually parse it pretty easily, so the tools that support text can be pretty universal. Grep, sed, and awk can do a lot. But to do anything more, you’re always going to need some kind of specialized parsing, even for tab delimited or CSV data. So, let the terminal work with generic text and be “good enough”. As an example: you mentioned JSON and YAML and suggested a slimmed down YAML might be more powerful. I deal with columnar data with tens to hundreds of columns and thousands of lines. This wouldn’t work with either JSON or YAML… it’s too verbose. Instead I use (compressed) tab delimited text files for their ease of parsing in many languages. But those won’t work for many other use cases. With data exchange, there is never going to be one right sized option for all needs. If you can accept that, then the “right” option is to choose none of them.

jampekka 3y ago
I'm not saying that processes wouldn't be free to spit out raw ASCII or whatever if they so wanted, but there should be some sort of POSIX-type standard that would spec typical output formats (e.g. standard --yaml flag). I see no urgent need for formal sending of schemas. It's a disgrace that most standard tools don't have even an option for any machine readable output. Or when they do, they are ad-hoc and there are ad-hoc parser tools/flags for those. I.e. you need a lot of (badly implemented) specialized tools. You can grep, sed and awk YAML all you want. It's just ASCII (or UTF-8) text.
- mbreese 3y ago
  I’m trying to think about what kind of tools would benefit from such an option… An obvious first tool would be “ls” to produce an easier to parse output. From there, maybe a “grep” to search fields and “sort” to sort by fields. I guess I can think of many consumers or YAML-like data, but what would some of the producers be?
  - jampekka 3y ago
    ls is a good example. Getting the -l output would be very useful for many applications. And would simplify ls itself (e.g. sorting could be done in a separate generig util, which would work with other tools as well). Would also take care of the whitespace horrors. Many tools don't have "stable CLI interface", which makes them difficult to automate. Apt and git come to mind immediately (although git generally allows specifying formats). Some tools (e.g. ffprobe) support JSON output, which provides very easy "bindings" for it (the actual libav APIs are quite painful).

jampekka 3y ago

I'm not saying that processes wouldn't be free to spit out raw ASCII or whatever if they so wanted, but there should be some sort of POSIX-type standard that would spec typical output formats (e.g. standard --yaml flag). I see no urgent need for formal sending of schemas. It's a disgrace that most standard tools don't have even an option for any machine readable output. Or when they do, they are ad-hoc and there are ad-hoc parser tools/flags for those. I.e. you need a lot of (badly implemented) specialized tools. You can grep, sed and awk YAML all you want. It's just ASCII (or UTF-8) text.

mbreese 3y ago
I’m trying to think about what kind of tools would benefit from such an option… An obvious first tool would be “ls” to produce an easier to parse output. From there, maybe a “grep” to search fields and “sort” to sort by fields. I guess I can think of many consumers or YAML-like data, but what would some of the producers be?
- jampekka 3y ago
  ls is a good example. Getting the -l output would be very useful for many applications. And would simplify ls itself (e.g. sorting could be done in a separate generig util, which would work with other tools as well). Would also take care of the whitespace horrors. Many tools don't have "stable CLI interface", which makes them difficult to automate. Apt and git come to mind immediately (although git generally allows specifying formats). Some tools (e.g. ffprobe) support JSON output, which provides very easy "bindings" for it (the actual libav APIs are quite painful).