6 ms·

> That doesn't mean that we can't do better, but I'd argue that a monolithic shell with a strict contract between commands is not the way to build a sustainabl

by mongol 3y ago

> That doesn't mean that we can't do better, but I'd argue that a monolithic shell with a strict contract between commands is not the way to build a sustainable and future-proof ecosystem. I don't think it is far fetched to imagine that most popular command line tools will have support for JSON output within let's say 5 years. With that, I think nushell's value proposition becomes a whole lot stronger. Granted, there will never be a time when all tools integrate well, but I can see a critical mass evolve.

fiddlerwoaroof 3y ago
I think standardized JSON output makes the value of nushell weaker, not stronger: more structure on stdin and stdout (e.g. switching them to unix sockets instead of fifos and passing FDs around) means the shell composing commands can be less sophisticated. The only way a monolith like nushell adds value is if you compare it to the current standard of unstructured bytes everywhere.
pmarreck 3y ago
I absolutely think that treating data piped from or to another process as JSON by default, and treating data going back to the terminal as text (by transforming the JSON to tabular data etc.) is the way to go (also because it's backwards-compatible). To the point that I wanted to write some wrappers for the standard commands that automatically did all this via `jq`. I know commands can detect whether they're being piped to another command or to a terminal, since certain commands will automatically skip ANSI coloring when being piped elsewhere...
- 8n4vidtmkvmk 3y ago
  Those commands that strip color very often get it wrong too. I have to force color so damn often...
- follower 3y ago
  > I wanted to write some wrappers for the standard commands that automatically did all this via `jq`. If you're not already aware of it, you may wish to check out `jc`[0] which describes itself as a "CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq..." The `jc` documentation[1] & parser[2] for `ls` also demonstrates that reliable & cross-platform parsing of even "basic" commands can be non-trivial. [0] https://github.com/kellyjonbrazil/jc https://github.com/kellyjonbrazil/jc [1] https://kellyjonbrazil.github.io/jc/docs/parsers/ls https://kellyjonbrazil.github.io/jc/docs/parsers/ls [2] https://github.com/kellyjonbrazil/jc/blob/4cd721be8595db52b620cc26cd455d95bf56b85b/jc/parsers/ls.py#L187 https://github.com/kellyjonbrazil/jc/blob/4cd721be8595db52b6...
  - pmarreck 3y ago
    This is interesting, but I like neither the TUI nor the fact that it's written in Python, lol. My idea was to have namespaced wrappers for the commands which were designed to generate output as JSON and/or accept input as JSON. So for example "ls" would have a wrapper "qls" (queryable ls) or maybe "jsls" (json LS), etc. But in thinking about it, many questions remain- Would some of the structs or struct elements have types, like "file_path" or "file_name" for example? Like for example they do: > $ jc dig example.com | jq -r '.[].answer[].data' and my API would be more like: > $ jsdig example.com -- '.[].answer[].data' (which would then pass those additional arguments to "jq -r" without having to pipe) The core idea is that you can have seamless integration of structured pipe data with regular piped data without having to change shells.
imiric 3y ago
Sure, but why JSON? :) What happens when a new format comes along that is better, more efficient, or whatever? Would all tools then need to be updated to support it? What if a change in the format is introduced? Would all tools need to be updated, and have to maintain backwards compatibility indefinitely? This would be an even bigger UX problem, while also adding much more maintenance work for tool developers. It might seem like unstructured text as an exchange format is a usability nuisance, but it is what makes independent and disparate tools work well together. Choosing a strict contract is only sustainable if a single project maintains all tools, and even then it's a huge maintenance effort.
- hermanradtke 3y ago
  > Would all tools then need to be updated to support it? The effort to translate JSON to another structured data format is orders of magnitude easier than converting the unstructured mess we have now into JSON. So, it does not matter if we choose JSON, XML, TOML, etc. Once we get structured data, then things progress quickly. The nice thing about JSON at this moment is that it is a very common and easy to use format.
- pmarreck 3y ago
  > Choosing a strict contract is only sustainable if a single project maintains all tools You're obviously not working in the web space, where literally hundreds of thousands of sites have somehow successfully standardized on interoperating via JSON input/output for their HTTP API endpoints despite using many, many different technology stacks, over the last 10 years or so. It isn't perfect, but it's easy to use and common and the tooling is mature.
- mongol 3y ago
  JSON seems to have established itself for this to a large extent already. I can imagine a binary format like MsgPack to work well for this too but I think JSON is good enough and popular enough that it could actually be embraced widely.

by mongol 3y ago

fiddlerwoaroof 3y ago

I think standardized JSON output makes the value of nushell weaker, not stronger: more structure on stdin and stdout (e.g. switching them to unix sockets instead of fifos and passing FDs around) means the shell composing commands can be less sophisticated. The only way a monolith like nushell adds value is if you compare it to the current standard of unstructured bytes everywhere.

pmarreck 3y ago

I absolutely think that treating data piped from or to another process as JSON by default, and treating data going back to the terminal as text (by transforming the JSON to tabular data etc.) is the way to go (also because it's backwards-compatible). To the point that I wanted to write some wrappers for the standard commands that automatically did all this via `jq`. I know commands can detect whether they're being piped to another command or to a terminal, since certain commands will automatically skip ANSI coloring when being piped elsewhere...

8n4vidtmkvmk 3y ago
Those commands that strip color very often get it wrong too. I have to force color so damn often...
follower 3y ago
> I wanted to write some wrappers for the standard commands that automatically did all this via `jq`. If you're not already aware of it, you may wish to check out `jc`[0] which describes itself as a "CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq..." The `jc` documentation[1] & parser[2] for `ls` also demonstrates that reliable & cross-platform parsing of even "basic" commands can be non-trivial. [0] https://github.com/kellyjonbrazil/jc https://github.com/kellyjonbrazil/jc [1] https://kellyjonbrazil.github.io/jc/docs/parsers/ls https://kellyjonbrazil.github.io/jc/docs/parsers/ls [2] https://github.com/kellyjonbrazil/jc/blob/4cd721be8595db52b620cc26cd455d95bf56b85b/jc/parsers/ls.py#L187 https://github.com/kellyjonbrazil/jc/blob/4cd721be8595db52b6...
- pmarreck 3y ago
  This is interesting, but I like neither the TUI nor the fact that it's written in Python, lol. My idea was to have namespaced wrappers for the commands which were designed to generate output as JSON and/or accept input as JSON. So for example "ls" would have a wrapper "qls" (queryable ls) or maybe "jsls" (json LS), etc. But in thinking about it, many questions remain- Would some of the structs or struct elements have types, like "file_path" or "file_name" for example? Like for example they do: > $ jc dig example.com | jq -r '.[].answer[].data' and my API would be more like: > $ jsdig example.com -- '.[].answer[].data' (which would then pass those additional arguments to "jq -r" without having to pipe) The core idea is that you can have seamless integration of structured pipe data with regular piped data without having to change shells.

imiric 3y ago

Sure, but why JSON? :) What happens when a new format comes along that is better, more efficient, or whatever? Would all tools then need to be updated to support it? What if a change in the format is introduced? Would all tools need to be updated, and have to maintain backwards compatibility indefinitely? This would be an even bigger UX problem, while also adding much more maintenance work for tool developers. It might seem like unstructured text as an exchange format is a usability nuisance, but it is what makes independent and disparate tools work well together. Choosing a strict contract is only sustainable if a single project maintains all tools, and even then it's a huge maintenance effort.

hermanradtke 3y ago
> Would all tools then need to be updated to support it? The effort to translate JSON to another structured data format is orders of magnitude easier than converting the unstructured mess we have now into JSON. So, it does not matter if we choose JSON, XML, TOML, etc. Once we get structured data, then things progress quickly. The nice thing about JSON at this moment is that it is a very common and easy to use format.
pmarreck 3y ago
> Choosing a strict contract is only sustainable if a single project maintains all tools You're obviously not working in the web space, where literally hundreds of thousands of sites have somehow successfully standardized on interoperating via JSON input/output for their HTTP API endpoints despite using many, many different technology stacks, over the last 10 years or so. It isn't perfect, but it's easy to use and common and the tooling is mature.
mongol 3y ago
JSON seems to have established itself for this to a large extent already. I can imagine a binary format like MsgPack to work well for this too but I think JSON is good enough and popular enough that it could actually be embraced widely.