8 ms·

> This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl

> This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting. How are Python/Ruby/Perl going to give you structured objects from "ps"? That's the promise of object-based pipes. You can get useful records out of your command-line tools without having to write an ad-hoc regex.

rbanffy 10y ago
> How are Python/Ruby/Perl going to give you structured objects from "ps"? The usual answer would be "by parsing /proc instead and returning meaningful objects that expose the relevant parts of the OS" but why would you want to do that when you could just output the relevant data in easy to parse XML or JSON?
mbrumlow 10y ago
I find that to be the issue. You are considering it just RAW text when it is actually formatted text that has been parsable for years with common unix command line tools. It not being in the format you consider a structured object does not mean it's not a object or even parsable. If you are using ad hoc regex I suspect you are not using all the tools available to you. I feel like Kernighan and Pike do a much better job of explaining than I could ever. https://www.amazon.com/Unix-Programming-Environment-Prentice-Hall-Software/dp/013937681X/ref=sr_1_1?ie=UTF8&qid=1471543239&sr=8-1&keywords=The+UNIX+Programming+Environment https://www.amazon.com/Unix-Programming-Environment-Prentice...
- madmax96 10y ago
  So much this. What the M$ community fails to see is that text streams can be consumed by __everyone__. We've basically all accepted the fact that programs are used longer and in different ways than we'd expect. A brittle object model can only lead to vendor lock in and indigestible output.
  - ygra 10y ago
    Nothing prevents me from changing those objects into a text stream. In fact, it's infinitely easier than turning a text stream into objects.
    - mbrumlow 10y ago
      I feel you missed my point. It's not just a stream of raw data. It's a stream of formatted text... There is no magic or hand waving involved.
  - temac 10y ago
    It's even more than that: text formatting allows the use of generic filtering and text processing tools. Whereas if you are using objects, you will tend to use less on-the-fly command line composition, and write or reuse more tools dedicated to one particular object or the other. In the end I'm not sure keeping the structuring as the default use case yields an usage so much improved in the real world, because if you work on a broad number of types you will tend to need to know more specialised tools, instead of generic ones, and have less higher-order composition tools at your disposal. Now of course you can always format to text from your structured object, but this does not matter. What matters is what is convenient in the proposed UI, and what is mainly practiced in the real world because of such convenience and the amplification loop it creates between tools authors and users. Also some objects are originally handled in text form, and their structured decomposition is extremely complex compared to an informal description and the naturally refinable heuristic which comes with it. For example you can grep into a source code in any language (with more or less efficiency depending on the language, but at least if there is a unique symbol you will find it), whereas trying to get and handle it in a structured way basically means you would need half of a (modular) compiler, and a huge manual to describe the structure, and possibly a non trivial amount of code to actually do the lookup. The PowerShell approach is not all-bad, though, and obviously there are some usages where it superior to text based shells. But for a day to day usage as a general purpose shell, and a programmer shell, I'll stick to the Unix approach.
    - Firadeoclus 10y ago
- haberman 10y ago
  > You are considering it just RAW text when it is actually formatted text that has been parsable for years with common unix command line tools. Parsing command output with sed/awk/etc (ie. "common unix command line tools") is absolutely an ad hoc parser. Let me give you an example that I recently ran into. I have a tool that parses the output of "readelf -sW", which dumps symbols and their sizes. The output normally looks like this: 885: 000000000043f0a0 249 FUNC WEAK DEFAULT 13 _ZNSt6vectorIPN3re23DFA5StateESaIS3_EE19_M_emplace_back_auxIJRKS3_EEEvDpOT_ 886: 000000000041c380 64 FUNC GLOBAL DEFAULT 13 _dwarf_get_return_address_reg 887: 0000000000424e60 122 FUNC GLOBAL DEFAULT 13 _dwarf_decode_s_leb128 888: 000000000043dca0 157 FUNC GLOBAL DEFAULT 13 _ZN3re23DFA10StateSaverC2EPS0_PNS0_5StateE So I wrote a regex to parse this. Seems pretty straightforward, right? But then I noticed a bug where some symbols were not showing up. And it turns out those symbols look like this: 5898: 00000000001a4d80 0x801058 OBJECT GLOBAL DEFAULT 33 _ZN8tcmalloc6Static9pageheap_E Notice the difference? Because it's a large symbol, readelf decided to print it starting with "0x" and in hex instead of decimal. I had to update my regex to accommodate this. That is what makes a parser "ad hoc". You write a parser based on the examples you have seen, but other examples might break your parser. Parsing text robustly is non-trivial. Worse, it is an unnecessary cognitive burden. Readelf already had this data in a structured format, why does it have to go to text and back? Why do I have to spend mental cycles figuring out which "common unix command-line tools" (and what options) can parse it back into a data structure?
  - mbrumlow 10y ago
    I happen to find your example of why to use a object a bit hilarious. You are right, readelf has the object in a structure -- because that is what elf is... typedef struct { uint32_t st_name; unsigned char st_info; unsigned char st_other; uint16_t st_shndx; Elf64_Addr st_value; uint64_t st_size; } Elf64_Sym; If you wanted the object why did you need readelf in the first place? Why not just read the elf format directly and bypass readelf all together? That seems to be what you are advocating by having readelf passing a object instead of what it does today.
Too 10y ago
For the most common use cases there would be a standard function to get structured objects, in Python you have os.listdir() instead of ls, etc. If there is none then you can call ps and parse the result manually or call the system functions with c interop.

rbanffy 10y ago

> How are Python/Ruby/Perl going to give you structured objects from "ps"? The usual answer would be "by parsing /proc instead and returning meaningful objects that expose the relevant parts of the OS" but why would you want to do that when you could just output the relevant data in easy to parse XML or JSON?

mbrumlow 10y ago

I find that to be the issue. You are considering it just RAW text when it is actually formatted text that has been parsable for years with common unix command line tools. It not being in the format you consider a structured object does not mean it's not a object or even parsable. If you are using ad hoc regex I suspect you are not using all the tools available to you. I feel like Kernighan and Pike do a much better job of explaining than I could ever. https://www.amazon.com/Unix-Programming-Environment-Prentice-Hall-Software/dp/013937681X/ref=sr_1_1?ie=UTF8&qid=1471543239&sr=8-1&keywords=The+UNIX+Programming+Environment https://www.amazon.com/Unix-Programming-Environment-Prentice...

madmax96 10y ago
So much this. What the M$ community fails to see is that text streams can be consumed by __everyone__. We've basically all accepted the fact that programs are used longer and in different ways than we'd expect. A brittle object model can only lead to vendor lock in and indigestible output.
- ygra 10y ago
  Nothing prevents me from changing those objects into a text stream. In fact, it's infinitely easier than turning a text stream into objects.
  - mbrumlow 10y ago
    I feel you missed my point. It's not just a stream of raw data. It's a stream of formatted text... There is no magic or hand waving involved.
- temac 10y ago
  It's even more than that: text formatting allows the use of generic filtering and text processing tools. Whereas if you are using objects, you will tend to use less on-the-fly command line composition, and write or reuse more tools dedicated to one particular object or the other. In the end I'm not sure keeping the structuring as the default use case yields an usage so much improved in the real world, because if you work on a broad number of types you will tend to need to know more specialised tools, instead of generic ones, and have less higher-order composition tools at your disposal. Now of course you can always format to text from your structured object, but this does not matter. What matters is what is convenient in the proposed UI, and what is mainly practiced in the real world because of such convenience and the amplification loop it creates between tools authors and users. Also some objects are originally handled in text form, and their structured decomposition is extremely complex compared to an informal description and the naturally refinable heuristic which comes with it. For example you can grep into a source code in any language (with more or less efficiency depending on the language, but at least if there is a unique symbol you will find it), whereas trying to get and handle it in a structured way basically means you would need half of a (modular) compiler, and a huge manual to describe the structure, and possibly a non trivial amount of code to actually do the lookup. The PowerShell approach is not all-bad, though, and obviously there are some usages where it superior to text based shells. But for a day to day usage as a general purpose shell, and a programmer shell, I'll stick to the Unix approach.
  - Firadeoclus 10y ago
haberman 10y ago
> You are considering it just RAW text when it is actually formatted text that has been parsable for years with common unix command line tools. Parsing command output with sed/awk/etc (ie. "common unix command line tools") is absolutely an ad hoc parser. Let me give you an example that I recently ran into. I have a tool that parses the output of "readelf -sW", which dumps symbols and their sizes. The output normally looks like this: 885: 000000000043f0a0 249 FUNC WEAK DEFAULT 13 _ZNSt6vectorIPN3re23DFA5StateESaIS3_EE19_M_emplace_back_auxIJRKS3_EEEvDpOT_ 886: 000000000041c380 64 FUNC GLOBAL DEFAULT 13 _dwarf_get_return_address_reg 887: 0000000000424e60 122 FUNC GLOBAL DEFAULT 13 _dwarf_decode_s_leb128 888: 000000000043dca0 157 FUNC GLOBAL DEFAULT 13 _ZN3re23DFA10StateSaverC2EPS0_PNS0_5StateE So I wrote a regex to parse this. Seems pretty straightforward, right? But then I noticed a bug where some symbols were not showing up. And it turns out those symbols look like this: 5898: 00000000001a4d80 0x801058 OBJECT GLOBAL DEFAULT 33 _ZN8tcmalloc6Static9pageheap_E Notice the difference? Because it's a large symbol, readelf decided to print it starting with "0x" and in hex instead of decimal. I had to update my regex to accommodate this. That is what makes a parser "ad hoc". You write a parser based on the examples you have seen, but other examples might break your parser. Parsing text robustly is non-trivial. Worse, it is an unnecessary cognitive burden. Readelf already had this data in a structured format, why does it have to go to text and back? Why do I have to spend mental cycles figuring out which "common unix command-line tools" (and what options) can parse it back into a data structure?
- mbrumlow 10y ago
  I happen to find your example of why to use a object a bit hilarious. You are right, readelf has the object in a structure -- because that is what elf is... typedef struct { uint32_t st_name; unsigned char st_info; unsigned char st_other; uint16_t st_shndx; Elf64_Addr st_value; uint64_t st_size; } Elf64_Sym; If you wanted the object why did you need readelf in the first place? Why not just read the elf format directly and bypass readelf all together? That seems to be what you are advocating by having readelf passing a object instead of what it does today.

Too 10y ago

For the most common use cases there would be a standard function to get structured objects, in Python you have os.listdir() instead of ls, etc. If there is none then you can call ps and parse the result manually or call the system functions with c interop.