3 ms·
> Pipes are wonderful! It's wonderful only if compared to worse things, pretending that PowerShell is not a thing, and that Python doesn't exist. UNIX pipes a
by jiggawatts 6y ago
> Pipes are wonderful!
It's wonderful only if compared to worse things, pretending that PowerShell is not a thing, and that Python doesn't exist.
UNIX pipes are a stringly-typed legacy that we've inherited from the 1970s. The technical constraints of its past have been internalised by its proponents, and lauded as benefits.
To put it most succinctly, the "byte stream" nature of UNIX pipes means that any command that does anything high-level such as processing "structured" data must have a parsing step on its input, and then a serialization step after its internal processing.
The myth of UNIX pipes is that it works like this:
process1 | process2 | process3 | ...
The reality is that physically, it actually works like this:
(process1,serialize) | (parse,process2,serialize) | ...
Where each one of those "parse" and "serialise" steps is unique and special, inflexible, and poorly documented. This is forced by the use of byte streams to connect each step. It cannot be circumvented! It's an inherent limitation of UNIX style pipes.
This is not a limitation of PowerShell, which is object oriented, and passes strongly typed, structured objects between processing steps. It makes it much more elegant, flexible, and in effect "more UNIX than UNIX".
If you want to see this in action, check out my little "challenge" that I posed in a similar thread on YC recently:
https://news.ycombinator.com/item?id=23257776 https://news.ycombinator.com/item?id=23257776
The solution provided by "JoshuaDavid" really impressed me, because I was under the impression that that simple task is actually borderline impossible with UNIX pipes and GNU tools:
https://news.ycombinator.com/item?id=23267901 https://news.ycombinator.com/item?id=23267901
Compare that to the much simpler equivalent in PowerShell:
https://news.ycombinator.com/item?id=23270291 https://news.ycombinator.com/item?id=23270291
Especially take note half of that script is sample data!
> Unix is seriously uncool with young people at the moment. I intend to turn that around and articles like this offer good material.
UNIX is entirely too cool with young people. They have conflated the legacy UNIX design of the 1970s with their preferred open source software, languages, and platforms.
There are better things out there, and UNIX isn't the be all and end all of system management.
- matheusmoreira 6y ago> Where each one of those "parse" and "serialise" steps is unique and special, inflexible, and poorly documented. This can also be a security concern. According to research, a great number of defects with security implications occur at the input handling layers: http://langsec.org http://langsec.org
- gorgoiler 6y agoTyped streams sound great. Most data being line oriented (with white space field separators) has historically worked well enough, but I get your point that the wheels will eventually fall off. It’s important to remember that the shell hasn’t always been about high concept programming tasks. $ grep TODO xmas-gifts grandma TODO auntie Sian TODO jeanie MASTODON T shirt The bug (bugs!) in the above example doesn’t really matter in the context of the task at hand.
- scbrg 6y agoThe reality is that physically, it actually works like this: (process1,serialize) | (parse,process2,serialize) | ... But as soon as you involve the network or persistent storage, you need to do all that anyway. And one of the beauties is that the tools are agnostic to where the data comes from, or goes.
- jiggawatts 6y agoSo you're saying optional things should be forced on everything, because the rare optional case needs it anyway? Look up what the Export-Csv, ConvertTo-Json, and Export-CliXml, Invoke-Command -ComputerName 'someserver', or Import-Sql commands do in PowerShell. Actually, never mind, the clear and consistent naming convention already told you exactly what they do: they serialise the structured pipeline objects into a variety of formats of your choosing. They can do this with files, in-memory, to and from the network, or even involving databases. This is the right way to do things. This is the UNIX philosophy. It's just that traditional shells typically used on UNIX do a bad job at implementing the philosophy.
- rcxdude 6y agoIf you have network or persistent storage, those parse and serialise steps are better served by a consistent, typed, and common serialize and parse step instead of a different ad-hoc step for each part of the chain (especially if there are then parts of the chain which are just munging the output of one serialisation step so it can work with the parse stage of another).