4 ms·

I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than

by nojvek 8y ago

I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects and you can get the field you want by piping to `Get-Item` or sum the file sizes, or filter only directories. It’s very expressive once you’re manipulating streams of objects with properties.

sevensor 8y ago
If you're using cut and awk to get a field out of ls, maybe what you want is actually stat(1)?
nijaru 8y ago
She is actually employed at GitHub now. https://twitter.com/jessfraz https://twitter.com/jessfraz
- schwartzworld 8y ago
  Which is owned by...
- lordgrenville 8y ago
  Still MSFT, in a way!
marpstar 8y ago
I've been following her on twitter for awhile. She went to MSFT only a couple years ago. From what I understand her expertise is in Linux.
npongratz 8y ago
> In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. I'm guessing you've mentioned using `ls` as a simple, first-thing-that-comes-to-mind example, which is cool. I just wanted to point out that if a person is piping ls's output, there are probably other, far better alternatives, such as (per the link below) `find` and globs: https://unix.stackexchange.com/a/247973 https://unix.stackexchange.com/a/247973 That has been my experience, at least.
hk__2 8y ago
Don’t you have to rewrite every single program to make it able to read those “objects”?
- ghusbands 8y ago
  You don't have to rewrite them; just write them. Every new OS/language needs its tooling/libraries to be written.
iamwil 8y ago
What happens when there are upstream changes to the objects? Does everything downstream just need to change, and hence, the upstream objects returned by programs need to be changed with care? Or is it using something like protobuf, where fields are only additive, but never deleted, for backwards compatibility? Or are the resulting chain of pipes so short lived, it doesn't matter?
- dan-robertson 8y ago
  What happens when you rely on non standard behaviour of unix tools, or just non-posix tools that change from beneath you? I’m not saying that this makes powershell pipes better/worse, just that this problem isn’t unique. Microsoft tends to be reasonably committed to backwards compatibility but I don’t know the answer to the question
  - Pawamoy 8y ago
    I think they were talking about objects at execution time, not version compatibility.
    - dan-robertson 8y ago
      I don’t really understand what this means. The parent specifically writes “backwards compatibility.” The only other thing I think you might be referring to is if the reading process mutates the objects does that somehow affect the writing end of the pipe but I think that doesn’t make sense from a “sane way of doing inter-process communication” standpoint. Is there something else you are referring to? Could you elaborate please?
- teyc 8y ago
  Powershell is a shell language, everything is dynamic, and does not error when you access a non-existent property on an object.
deleted 8y ago
[deleted]
paavoova 8y ago
The downside of objects is performance due to the e.g. increased overhead a file object carries. Plus the exact same can be argued with everything being bytes/text on Unix, in that it make everything simple but versatile. > get some field out of `ls` If you're parsing the output of ls, you're doing it wrong. Filenames can contain characters such as newlines, etc, and so you generally want to use globbing/other shell builtins, or utils like find with null as the separator instead of newline.
anthk 8y ago
Never parse ls.
- enriquto 8y ago
  > Never parse ls. I have heard this several times, but either I do not understand it or I disagree. Do you mean parsing the output of the ls program? Parsing ls output is not wrong, the program produces a text stream that is easy and useful to parse. There's nothing to be ashamed when doing it, even when you can do it in a different, even shorter way. I do grep over ls output daily, and I find it much more convenient than writing wildcards.
  - xelxebar 8y ago
    One can certainly do fine by grepping ls output in one-off instances, but I'd be really hesitant to put that in a script. For given paths, stat command essentially lets us directly access their inode structs, and invocations are nicely concise. The find util then lets us select files based on inode fields. Both tools do take a bit of learning, but considerably less than grep and regexs. Anyway, I've personally found find and stat to be really nice and ergonomic after the initial learning period.
flukus 8y ago
> Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. Unix pipes can pipe objects, binary data, text, json encoded data, etc. The problem is it adds a lot of complication and simply doesn't offer much in practice, so text is still widely used and binary data in a few specific cases.

by nojvek 8y ago

sevensor 8y ago

If you're using cut and awk to get a field out of ls, maybe what you want is actually stat(1)?

nijaru 8y ago

She is actually employed at GitHub now. https://twitter.com/jessfraz https://twitter.com/jessfraz

schwartzworld 8y ago
Which is owned by...
lordgrenville 8y ago
Still MSFT, in a way!

marpstar 8y ago

I've been following her on twitter for awhile. She went to MSFT only a couple years ago. From what I understand her expertise is in Linux.

npongratz 8y ago

> In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. I'm guessing you've mentioned using `ls` as a simple, first-thing-that-comes-to-mind example, which is cool. I just wanted to point out that if a person is piping ls's output, there are probably other, far better alternatives, such as (per the link below) `find` and globs: https://unix.stackexchange.com/a/247973 https://unix.stackexchange.com/a/247973 That has been my experience, at least.

hk__2 8y ago

Don’t you have to rewrite every single program to make it able to read those “objects”?

ghusbands 8y ago
You don't have to rewrite them; just write them. Every new OS/language needs its tooling/libraries to be written.

iamwil 8y ago

What happens when there are upstream changes to the objects? Does everything downstream just need to change, and hence, the upstream objects returned by programs need to be changed with care? Or is it using something like protobuf, where fields are only additive, but never deleted, for backwards compatibility? Or are the resulting chain of pipes so short lived, it doesn't matter?

dan-robertson 8y ago
What happens when you rely on non standard behaviour of unix tools, or just non-posix tools that change from beneath you? I’m not saying that this makes powershell pipes better/worse, just that this problem isn’t unique. Microsoft tends to be reasonably committed to backwards compatibility but I don’t know the answer to the question
- Pawamoy 8y ago
  I think they were talking about objects at execution time, not version compatibility.
  - dan-robertson 8y ago
    I don’t really understand what this means. The parent specifically writes “backwards compatibility.” The only other thing I think you might be referring to is if the reading process mutates the objects does that somehow affect the writing end of the pipe but I think that doesn’t make sense from a “sane way of doing inter-process communication” standpoint. Is there something else you are referring to? Could you elaborate please?
teyc 8y ago
Powershell is a shell language, everything is dynamic, and does not error when you access a non-existent property on an object.

deleted 8y ago

[deleted]

paavoova 8y ago

The downside of objects is performance due to the e.g. increased overhead a file object carries. Plus the exact same can be argued with everything being bytes/text on Unix, in that it make everything simple but versatile. > get some field out of `ls` If you're parsing the output of ls, you're doing it wrong. Filenames can contain characters such as newlines, etc, and so you generally want to use globbing/other shell builtins, or utils like find with null as the separator instead of newline.

anthk 8y ago

Never parse ls.

enriquto 8y ago
> Never parse ls. I have heard this several times, but either I do not understand it or I disagree. Do you mean parsing the output of the ls program? Parsing ls output is not wrong, the program produces a text stream that is easy and useful to parse. There's nothing to be ashamed when doing it, even when you can do it in a different, even shorter way. I do grep over ls output daily, and I find it much more convenient than writing wildcards.
- xelxebar 8y ago
  One can certainly do fine by grepping ls output in one-off instances, but I'd be really hesitant to put that in a script. For given paths, stat command essentially lets us directly access their inode structs, and invocations are nicely concise. The find util then lets us select files based on inode fields. Both tools do take a bit of learning, but considerably less than grep and regexs. Anyway, I've personally found find and stat to be really nice and ergonomic after the initial learning period.

flukus 8y ago

> Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. Unix pipes can pipe objects, binary data, text, json encoded data, etc. The problem is it adds a lot of complication and simply doesn't offer much in practice, so text is still widely used and binary data in a few specific cases.