8 ms·

“Cmdlets return objects not text, making it easier to access specific components of the returned object” This is for me the key value of shells like PowerShell

“Cmdlets return objects not text, making it easier to access specific components of the returned object” This is for me the key value of shells like PowerShell. Uncountable billions of person hours have been wasted parsing the text output of common *nix commands. It’s time to move away from the “everything is text with random formatting” philosophy.

bheadmaster 4y ago
That only works if there is a type of object for each and every type of output you will ever need. In my experience, there are often commands which just output text, and I need to parse them somehow, which leads to an awkward flood of Out-String | { $_ | Select-String -Pattern "etc." } type commands which are hard for me to get right. Perhaps that's just a reflection of my lack of experience with PowerShell, but assuming commands have a stable CLI interface (which most commands do in the *nix world), it's much easier for me to simply parse things with sed/awk/grep.
- Pet_Ant 4y ago
  > That only works if there is a type of object for each and every type of output you will ever need. Everything returned by the built-in commands is. If you are calling some executable, yeah, then you'll need to wrap it. I do recommend wrapping commonly used executables with things that return PowerShell objects. Tedious at first but as you build up over time you find yourself straying into string parsing less and less.
  - crabbone 4y ago
    > I do recommend wrapping commonly used executables with things that return PowerShell objects. I'd better spend my time writing a different shell. This sounds like a life-long with ever increasing number of bugs nightmare to support kind of project.
    - injidup 4y ago
      And what would your different shell do with a command that just returned plain text. You have the same parsing problem. Powershell gives you a standard object system to work with that if you lift stuff into are then available to everything else. With text shells everyone ends up writing a different parser each time for each small script.
      - bheadmaster 4y ago
        > And what would your different shell do with a command that just returned plain text It would give me tools like sed/grep/awk/tr/cut/etc., and allow easy capturing of output into variables and performing bash substitutions. IMHO, that's much more powerful than PowerShell's Select-String.
        AnIdiotOnTheNet 4y ago
        ...its the exact same damn thing. Parse text, put it into variables. PowerShell has the equivalent of all those same tools. The big difference is that you don't have to use them for every goddamned command.
        bheadmaster 4y ago
        > PowerShell has the equivalent of all those same tools. Really? What are the equivalents?
        majkinetor 4y ago
        Select-String and Select-Object cover all those. Also, you can use linux tools in PowerShell, but that is not the point, it is back to stone age moment. Shell scripts shouldn't be other name for text parsing.
        bheadmaster 4y ago
        Select-String covers grep (regex match) and Select-Object doesn't do anything for text parsing. You cannot use linux tools in PowerShell in most cases, since they aren't installed by default. > Shell scripts shouldn't be other name for text parsing. That may be true, but like any other ideal - it's just that, and ideal. In the real world a lot of shell scripting requires parsing unstructured text from various command outputs and files. PowerShell works great in its own microcosm of objects and commands that manipulate objects, but as soon as you step out of that comfort zone it becomes awkward to use.
- majkinetor 4y ago
  > Perhaps that's just a reflection of my lack of experience with PowerShell Obviously. If you create pwsh script that is longer then equivalent bash script and more confusing, then you are doing it wrong. One almost never parses string in pwsh, unless string parsing is your primary task itself. Contrary, in bash, you always parse text, no matter what you want to do, which is defocusing and hard to maintain, and has cross-OS problems even in *nix land (solaris vs red had comes to mind and their respective ls/stat output which differs).
  - bheadmaster 4y ago
    > One almost never parses string in pwsh Assuming that there are object-based commands that do everything that you need, which is quite a generous assumption.
dangus 4y ago
I think then the question becomes "Why use PowerShell rather than other scripting languages that give you objects?" I'm certainly not saying PowerShell is bad to use but if you aren't on Windows you're probably already scripting in Ruby or Python or something like that. Then if your script becomes more of a serious thing, all the stuff you're interacting with probably has a Python SDK and you'll never find that with PowerShell.
crabbone 4y ago
The word "easier" in your quote is a lie. Working with text output is easier because it's uniform, conceptually simple and easy to debug. Working with hierarchies of objects is a lot more complicated. Comparing things becomes a nightmare. Working with unknown hierarchies becomes a nightmare. It's a lot harder to work with objects. But, in some situations it's more rewarding because it eliminates certain classes of errors. I.e. you put more work in, and get arguably better results. The problem with PowerShell approach is that it's attempting to do it on the wrong level. The operating system and the programs it runs have to understand structured input and produce structured output. The system also needs to provide serialization protocol so that multiple systems can communicate with each other not breaking the illusion that they are using objects. However, object RPC is not a solved problem... In other words, PowerShell is a pretension. It solves some easy problems and if you don't give it enough thought, you may think that it will solve harder problems too... but it turns out there's some fundamental problem with this approach. So, it's OK to use it as a gimmick, but I would never consider using it as a replacement of Unix shell (no matter how much I hate it).
- oblio 4y ago
  Why are you working with hierarchies of objects? Are we back in 90s C++ land? PowerShell objects are flat, 99.99% of the time. And when you say compare, you don't compare objects, also 99.99% of the time. You compare attributes.
  - crabbone 4y ago
    I don't care if they are flat 99.99% of the time. Also, of course your statistic is invented on the spot and not warranted by any kind of research. Even a slight possibility of hierarchy is a no-go. Because someone will use it. And then it will create a new nightmare of dependencies. If I wanted that, I'd be using Java.
- AnIdiotOnTheNet 4y ago
  > Working with text output is easier because it's uniform If it is so uniform, why does every command need a dozen switches for modifying their output? Why do we need to use complicated tools like awk to reformat it for input into other commands? Everything you cite as a problem with structured IO is a problem with text too, it just happens to also suffer from in-band signaling issues and be human readable. Oh and of course it doesn't have a standardized structure at all.
squarefoot 4y ago
Most *nix commands already output binary where needed. Text is just how they're interpreted by the terminal because the terminal understands only text, but when you for example redirect a command into another command, if the 1st one spits out binary, the 2nd one gets binary. Typing cat /dev/urandom garbles the terminal because it's binary and the terminal doesn't know how to print it, but cat /dev/urandom|hd prints just fine because hd turns binary data into a ascii hexdump. A program will be more comfortable with reading the 1st, a human very likely the 2nd.
throw0101c 4y ago
Or make it flexible: > libxo - A Library for Generating Text, XML, JSON, and HTML Output * https://github.com/Juniper/libxo/ https://github.com/Juniper/libxo/ * https://wiki.freebsd.org/LibXo https://wiki.freebsd.org/LibXo Want structure? Ask for JSON or XML and parse. Otherwise it's the regular text output.

bheadmaster 4y ago

That only works if there is a type of object for each and every type of output you will ever need. In my experience, there are often commands which just output text, and I need to parse them somehow, which leads to an awkward flood of Out-String | { $_ | Select-String -Pattern "etc." } type commands which are hard for me to get right. Perhaps that's just a reflection of my lack of experience with PowerShell, but assuming commands have a stable CLI interface (which most commands do in the *nix world), it's much easier for me to simply parse things with sed/awk/grep.

Pet_Ant 4y ago
> That only works if there is a type of object for each and every type of output you will ever need. Everything returned by the built-in commands is. If you are calling some executable, yeah, then you'll need to wrap it. I do recommend wrapping commonly used executables with things that return PowerShell objects. Tedious at first but as you build up over time you find yourself straying into string parsing less and less.
- crabbone 4y ago
  > I do recommend wrapping commonly used executables with things that return PowerShell objects. I'd better spend my time writing a different shell. This sounds like a life-long with ever increasing number of bugs nightmare to support kind of project.
  - injidup 4y ago
    And what would your different shell do with a command that just returned plain text. You have the same parsing problem. Powershell gives you a standard object system to work with that if you lift stuff into are then available to everything else. With text shells everyone ends up writing a different parser each time for each small script.
    - bheadmaster 4y ago
      > And what would your different shell do with a command that just returned plain text It would give me tools like sed/grep/awk/tr/cut/etc., and allow easy capturing of output into variables and performing bash substitutions. IMHO, that's much more powerful than PowerShell's Select-String.
      - AnIdiotOnTheNet 4y ago
        ...its the exact same damn thing. Parse text, put it into variables. PowerShell has the equivalent of all those same tools. The big difference is that you don't have to use them for every goddamned command.
        bheadmaster 4y ago
        > PowerShell has the equivalent of all those same tools. Really? What are the equivalents?
        majkinetor 4y ago
        Select-String and Select-Object cover all those. Also, you can use linux tools in PowerShell, but that is not the point, it is back to stone age moment. Shell scripts shouldn't be other name for text parsing.
        bheadmaster 4y ago
        Select-String covers grep (regex match) and Select-Object doesn't do anything for text parsing. You cannot use linux tools in PowerShell in most cases, since they aren't installed by default. > Shell scripts shouldn't be other name for text parsing. That may be true, but like any other ideal - it's just that, and ideal. In the real world a lot of shell scripting requires parsing unstructured text from various command outputs and files. PowerShell works great in its own microcosm of objects and commands that manipulate objects, but as soon as you step out of that comfort zone it becomes awkward to use.
majkinetor 4y ago
> Perhaps that's just a reflection of my lack of experience with PowerShell Obviously. If you create pwsh script that is longer then equivalent bash script and more confusing, then you are doing it wrong. One almost never parses string in pwsh, unless string parsing is your primary task itself. Contrary, in bash, you always parse text, no matter what you want to do, which is defocusing and hard to maintain, and has cross-OS problems even in *nix land (solaris vs red had comes to mind and their respective ls/stat output which differs).
- bheadmaster 4y ago
  > One almost never parses string in pwsh Assuming that there are object-based commands that do everything that you need, which is quite a generous assumption.

dangus 4y ago

I think then the question becomes "Why use PowerShell rather than other scripting languages that give you objects?" I'm certainly not saying PowerShell is bad to use but if you aren't on Windows you're probably already scripting in Ruby or Python or something like that. Then if your script becomes more of a serious thing, all the stuff you're interacting with probably has a Python SDK and you'll never find that with PowerShell.

crabbone 4y ago

The word "easier" in your quote is a lie. Working with text output is easier because it's uniform, conceptually simple and easy to debug. Working with hierarchies of objects is a lot more complicated. Comparing things becomes a nightmare. Working with unknown hierarchies becomes a nightmare. It's a lot harder to work with objects. But, in some situations it's more rewarding because it eliminates certain classes of errors. I.e. you put more work in, and get arguably better results. The problem with PowerShell approach is that it's attempting to do it on the wrong level. The operating system and the programs it runs have to understand structured input and produce structured output. The system also needs to provide serialization protocol so that multiple systems can communicate with each other not breaking the illusion that they are using objects. However, object RPC is not a solved problem... In other words, PowerShell is a pretension. It solves some easy problems and if you don't give it enough thought, you may think that it will solve harder problems too... but it turns out there's some fundamental problem with this approach. So, it's OK to use it as a gimmick, but I would never consider using it as a replacement of Unix shell (no matter how much I hate it).

oblio 4y ago
Why are you working with hierarchies of objects? Are we back in 90s C++ land? PowerShell objects are flat, 99.99% of the time. And when you say compare, you don't compare objects, also 99.99% of the time. You compare attributes.
- crabbone 4y ago
  I don't care if they are flat 99.99% of the time. Also, of course your statistic is invented on the spot and not warranted by any kind of research. Even a slight possibility of hierarchy is a no-go. Because someone will use it. And then it will create a new nightmare of dependencies. If I wanted that, I'd be using Java.
AnIdiotOnTheNet 4y ago
> Working with text output is easier because it's uniform If it is so uniform, why does every command need a dozen switches for modifying their output? Why do we need to use complicated tools like awk to reformat it for input into other commands? Everything you cite as a problem with structured IO is a problem with text too, it just happens to also suffer from in-band signaling issues and be human readable. Oh and of course it doesn't have a standardized structure at all.

squarefoot 4y ago

Most *nix commands already output binary where needed. Text is just how they're interpreted by the terminal because the terminal understands only text, but when you for example redirect a command into another command, if the 1st one spits out binary, the 2nd one gets binary. Typing cat /dev/urandom garbles the terminal because it's binary and the terminal doesn't know how to print it, but cat /dev/urandom|hd prints just fine because hd turns binary data into a ascii hexdump. A program will be more comfortable with reading the 1st, a human very likely the 2nd.

throw0101c 4y ago

Or make it flexible: > libxo - A Library for Generating Text, XML, JSON, and HTML Output * https://github.com/Juniper/libxo/ https://github.com/Juniper/libxo/ * https://wiki.freebsd.org/LibXo https://wiki.freebsd.org/LibXo Want structure? Ask for JSON or XML and parse. Otherwise it's the regular text output.