7 ms·

I'm interested in progress in this area, but I'm disappointed the article seems to ignore the existence of powershell, which does some things different to tradi

I'm interested in progress in this area, but I'm disappointed the article seems to ignore the existence of powershell, which does some things different to traditional linux shells -- I would at least like a discussion of how this related to powershell.

rkangel 6y ago
It is worth noting that this article is from 2017.
- p_l 6y ago
  Which is over 5 years after Powershell became common.
  - INTPenis 6y ago
    Yes but some would argue Psh didn't become useful until release 6 in 2018.
    - p_l 6y ago
      That would be... an opinion. Not very grounded one unless one's whole base for it was "I couldn't use it on Linux".
    - indymike 6y ago
      As long as I have to write C# wrappers to launch commands like I would in unix with xargs or parallel in PowerShell it's still not the ticket.
      - TeMPOraL 6y ago
        You shouldn't need to, unless your PowerShell installation doesn't support Invoke-Command, or ForEach-Object -Parallel...
        indymike 6y ago
        Yeah... shouldn't need to but... there are issues with escaping command line parameters with invoke-command and ForEach-Object that prevent parallelizing older cmd-style command line applications. The work around is to write a wrapper in a compiled language and use PowerShell to invoke the wrapper... There's a lot of really cool stuff in PowerShell.
        TeMPOraL 6y ago
        > there are issues with escaping command line parameters with invoke-command and ForEach-Object that prevent parallelizing older cmd-style command line applications. Can you tell me more? So I know what to expect when I hit such issue myself. > The work around is to write a wrapper in a compiled language and use PowerShell to invoke the wrapper... Wonder if you need to really break out of PowerShell for this. Recently when doing some scripting at work, I ended up writing .NET code straight in the script. It looks like this: $someCSharp = @" using System; //... public class Foo : Bar { public override object Baz(Quux arg) { //... } } "@ Add-Type $someCSharp -ReferencedAssemblies C:\Path\To\Some\dependency-I-need.dll This makes PowerShell compile your code into an in-memory assembly and make it available in the session, just as if it was an external .NET type. You could probably make a module out of it. Hell, cmdlets are .NET types underneath, you can make new ones from C#, so you could probably make such ad-hoc .NET classes available as PowerShell cmdlets. I suppose using a proper .NET development environment for this would be more kosher; I arrived at the solution described above because my needs were simple, like binding .NET regex engine to SQLite (to get REGEXP operator in queries), or convincing Invoke-RestMethod to stop checking certificate validity. Still, I love the capability, the fact that I can just drop into C# in the middle of the script. It's one of those things that make me think of Lisp machines and Common Lisp the experience of being able to deeply interact with your environment, and the joy of writing "scripts" that are automatically AOT-compiled (like with SBCL implementation of Common Lisp).
      - p_l 6y ago
        And neither of those are native parts of shells but external applications, dare I say, just like the C# wrappers. (Especially parallel, since it's nowhere close to default install, and has very pesky license notice). OTOH, PS 7.0 added ForEach -Parallel
        indymike 6y ago
        Yeah, but & is a thing and will do the job nicely as long as you don't try to pipe output to the same file... also, you can invoke pretty much any command with xargs... the issue was escaping escaped stuff with PowerShell. Hopefully things have gotten better.
crazyloglad 6y ago
There was no point in bringing up powershell at the time, its legacy is an entirely different beast from vtXYZ and would invite the next "why isn't hyper.is covered" and "why isn't upterm covered and ..." The fabric under attack was the bastard display server protocol itself and not the parts that tie in with the strong points of Powershell, i.e. shell UI design or data exchange format specifics. This story was continued in a post from yesterday that fizzled, https://arcan-fe.com/2021/04/12/introducing-pipeworld/#pipelines https://arcan-fe.com/2021/04/12/introducing-pipeworld/#pipel...
yourapostasy 6y ago
The "key issues to address" list explicitly calls out many of the challenges Powershell manages with its object-based view of the world. It looks like Arcan's SHMIF layer is meant to let developers using Arcan capture rich data and metadata before it gets smooshed into the VTxxx emulation layer and smeared away in a sea of screen formatting characters. Where Arcan cannily (IMHO) differs from Powershell is it acknowledges the transition from existing toolchains will be organic instead of demanding wholesale operational changes to a different processing model. So it splits the data and metadata processing pipeline, exposes it at key points to the developers using Arcan, and lets an emulation layer cover legacy use cases, while enabling Arcan developers to create much richer data representations underneath the covers that can be tapped if desired by users via Lua manipulating the many different Arcan components (I think). This opens up hybrid GUI/TUI/other-interface best-of-breed/mixed-mode options that are unreasonably challenging to implement (especially with sync logic) with current UI stacks. I like this approach more than Powershell's, but it remains to be seen how Arcan handles the guidance of the data and metadata ontology and resultant taxonomy; it is a much deeper problem space than just defining a default namespace, a way to modularly plug into it with extensions, then standing back and letting developers have at it. The way Powershell handled it, introspection and discoverability are out of band instead of a batteries included experience, and module management is still split out of band between Nuget, PrivateGallery, custom solutions, hand-curated Git+Jenkins lash-ups, etc. For my small-scale, personal projects like for my family's intranet, the lack of an opinionated default for at-scale management of these issues is not a problem. But at my clients with dozens, hundreds and even thousands of developers, it presents software engineering management challenges. Solvable, just a developer experience hygiene factor I'd rather not deal with because solving it adds zero business value, it's just the cost of doing business.
donatj 6y ago
Is it possible to return structured data in PowerShell from non-.Net languages? I am genuinely asking here, it’s been years since I’ve poked it. If not though that seems like an enormous stumbling block.
- kaba0 6y ago
  Well, it can export as json, xml and I’m sure some other formats as well, so in principle it can.
  - rbanffy 6y ago
    That sounds interesting, but either programs deal with schema transforms, or we'll have to write a lot of that kind of logic in XSLT or jq (kind of the same stuff we do with cut, sed, awk, sort and grep for unstructured streams). Not sure it's that useful a feature unless the domain of the programs is narrow.
    - kaba0 6y ago
      To be honest, I’ve been thinking about a shell based on the JVM. Especially now with GraalVM, you can have truly polyglot programs running on top of it, with inlinig cross-language boundaries! But of course native code execution should be “bootstrapped” into that world somehow. (Actually, it can use LLVM IR and that as well can run a C function on js objects which is phenomenal, but people do want to run native software from a shell)
      - rbanffy 6y ago
        While you'll be able to invoke methods on returned objects, if you are piping stuff, you'll still need to deal with hammering the output of one program into the input of another.
    - TeMPOraL 6y ago
      I think it could be useful - from PowerShell's perspective, all you need is to wrap your "classical" executables to somehow accept and emit .NET objects, and you can carry on from there. Importantly, no more writing parsers in the pipeline or in your scripts. I could see a whole library of such wrappers being built for common executables (and new popular OSS CLI tools could provide their own wrapper along with the tool).
Koshkin 6y ago
To be fair, the article seems to be more about the terminal emulation and the limitations it puts on the CLI rather than about the shell language.
- rbanffy 6y ago
  Proper VT emulation gives you multiple fonts, mouse and windows.
pjmlp 6y ago
Powershell is great, it is the closest of the REPL experience from Xerox PARC, Lisp Machines and ETHZ workstations. The experience could be replicated in UNIX shells, but noone sems to care to actually go down that path.
- TeMPOraL 6y ago
  I feel that the closest we can get to a Lisp machine with modern components right now would be to get PowerShell, Emacs and StumpWM (or EXWM) to deeply interoperate.
- ticviking 6y ago
  I mean it's possible to run PowerShell on Linux, but PowerShell has some pretty significant rough edges, including a pretty painful performance story for something I want to use as a UI.
  - pjmlp 6y ago
    It is also a matter of culture, as mentioned you could have a UNIX shell being structured, able to directly interact with shared objects, IPC with running applications, yet very few do it.
marlowe221 6y ago
I work in a Windows shop (security compliance reasons for federal government contracts). Before I got this job I was pretty much only developing in Linux. Honestly? Powershell is pretty cool, though it feels a little strange to admit that. It has it's own very powerful syntax and virtually all of the common Bash commands are now aliased by default, so it's pretty smooth transition for the basic stuff.
- richardwhiuk 6y ago
  See https://github.com/PowerShell/PowerShell/pull/1901 https://github.com/PowerShell/PowerShell/pull/1901

rkangel 6y ago

It is worth noting that this article is from 2017.

p_l 6y ago
Which is over 5 years after Powershell became common.
- INTPenis 6y ago
  Yes but some would argue Psh didn't become useful until release 6 in 2018.
  - p_l 6y ago
    That would be... an opinion. Not very grounded one unless one's whole base for it was "I couldn't use it on Linux".
  - indymike 6y ago
    As long as I have to write C# wrappers to launch commands like I would in unix with xargs or parallel in PowerShell it's still not the ticket.
    - TeMPOraL 6y ago
      You shouldn't need to, unless your PowerShell installation doesn't support Invoke-Command, or ForEach-Object -Parallel...
      - indymike 6y ago
        Yeah... shouldn't need to but... there are issues with escaping command line parameters with invoke-command and ForEach-Object that prevent parallelizing older cmd-style command line applications. The work around is to write a wrapper in a compiled language and use PowerShell to invoke the wrapper... There's a lot of really cool stuff in PowerShell.
        TeMPOraL 6y ago
        > there are issues with escaping command line parameters with invoke-command and ForEach-Object that prevent parallelizing older cmd-style command line applications. Can you tell me more? So I know what to expect when I hit such issue myself. > The work around is to write a wrapper in a compiled language and use PowerShell to invoke the wrapper... Wonder if you need to really break out of PowerShell for this. Recently when doing some scripting at work, I ended up writing .NET code straight in the script. It looks like this: $someCSharp = @" using System; //... public class Foo : Bar { public override object Baz(Quux arg) { //... } } "@ Add-Type $someCSharp -ReferencedAssemblies C:\Path\To\Some\dependency-I-need.dll This makes PowerShell compile your code into an in-memory assembly and make it available in the session, just as if it was an external .NET type. You could probably make a module out of it. Hell, cmdlets are .NET types underneath, you can make new ones from C#, so you could probably make such ad-hoc .NET classes available as PowerShell cmdlets. I suppose using a proper .NET development environment for this would be more kosher; I arrived at the solution described above because my needs were simple, like binding .NET regex engine to SQLite (to get REGEXP operator in queries), or convincing Invoke-RestMethod to stop checking certificate validity. Still, I love the capability, the fact that I can just drop into C# in the middle of the script. It's one of those things that make me think of Lisp machines and Common Lisp the experience of being able to deeply interact with your environment, and the joy of writing "scripts" that are automatically AOT-compiled (like with SBCL implementation of Common Lisp).
    - p_l 6y ago
      And neither of those are native parts of shells but external applications, dare I say, just like the C# wrappers. (Especially parallel, since it's nowhere close to default install, and has very pesky license notice). OTOH, PS 7.0 added ForEach -Parallel
      - indymike 6y ago
        Yeah, but & is a thing and will do the job nicely as long as you don't try to pipe output to the same file... also, you can invoke pretty much any command with xargs... the issue was escaping escaped stuff with PowerShell. Hopefully things have gotten better.

crazyloglad 6y ago

There was no point in bringing up powershell at the time, its legacy is an entirely different beast from vtXYZ and would invite the next "why isn't hyper.is covered" and "why isn't upterm covered and ..." The fabric under attack was the bastard display server protocol itself and not the parts that tie in with the strong points of Powershell, i.e. shell UI design or data exchange format specifics. This story was continued in a post from yesterday that fizzled, https://arcan-fe.com/2021/04/12/introducing-pipeworld/#pipelines https://arcan-fe.com/2021/04/12/introducing-pipeworld/#pipel...

yourapostasy 6y ago

The "key issues to address" list explicitly calls out many of the challenges Powershell manages with its object-based view of the world. It looks like Arcan's SHMIF layer is meant to let developers using Arcan capture rich data and metadata before it gets smooshed into the VTxxx emulation layer and smeared away in a sea of screen formatting characters. Where Arcan cannily (IMHO) differs from Powershell is it acknowledges the transition from existing toolchains will be organic instead of demanding wholesale operational changes to a different processing model. So it splits the data and metadata processing pipeline, exposes it at key points to the developers using Arcan, and lets an emulation layer cover legacy use cases, while enabling Arcan developers to create much richer data representations underneath the covers that can be tapped if desired by users via Lua manipulating the many different Arcan components (I think). This opens up hybrid GUI/TUI/other-interface best-of-breed/mixed-mode options that are unreasonably challenging to implement (especially with sync logic) with current UI stacks. I like this approach more than Powershell's, but it remains to be seen how Arcan handles the guidance of the data and metadata ontology and resultant taxonomy; it is a much deeper problem space than just defining a default namespace, a way to modularly plug into it with extensions, then standing back and letting developers have at it. The way Powershell handled it, introspection and discoverability are out of band instead of a batteries included experience, and module management is still split out of band between Nuget, PrivateGallery, custom solutions, hand-curated Git+Jenkins lash-ups, etc. For my small-scale, personal projects like for my family's intranet, the lack of an opinionated default for at-scale management of these issues is not a problem. But at my clients with dozens, hundreds and even thousands of developers, it presents software engineering management challenges. Solvable, just a developer experience hygiene factor I'd rather not deal with because solving it adds zero business value, it's just the cost of doing business.

donatj 6y ago

Is it possible to return structured data in PowerShell from non-.Net languages? I am genuinely asking here, it’s been years since I’ve poked it. If not though that seems like an enormous stumbling block.

kaba0 6y ago
Well, it can export as json, xml and I’m sure some other formats as well, so in principle it can.
- rbanffy 6y ago
  That sounds interesting, but either programs deal with schema transforms, or we'll have to write a lot of that kind of logic in XSLT or jq (kind of the same stuff we do with cut, sed, awk, sort and grep for unstructured streams). Not sure it's that useful a feature unless the domain of the programs is narrow.
  - kaba0 6y ago
    To be honest, I’ve been thinking about a shell based on the JVM. Especially now with GraalVM, you can have truly polyglot programs running on top of it, with inlinig cross-language boundaries! But of course native code execution should be “bootstrapped” into that world somehow. (Actually, it can use LLVM IR and that as well can run a C function on js objects which is phenomenal, but people do want to run native software from a shell)
    - rbanffy 6y ago
      While you'll be able to invoke methods on returned objects, if you are piping stuff, you'll still need to deal with hammering the output of one program into the input of another.
  - TeMPOraL 6y ago
    I think it could be useful - from PowerShell's perspective, all you need is to wrap your "classical" executables to somehow accept and emit .NET objects, and you can carry on from there. Importantly, no more writing parsers in the pipeline or in your scripts. I could see a whole library of such wrappers being built for common executables (and new popular OSS CLI tools could provide their own wrapper along with the tool).

Koshkin 6y ago

To be fair, the article seems to be more about the terminal emulation and the limitations it puts on the CLI rather than about the shell language.

rbanffy 6y ago
Proper VT emulation gives you multiple fonts, mouse and windows.

pjmlp 6y ago

Powershell is great, it is the closest of the REPL experience from Xerox PARC, Lisp Machines and ETHZ workstations. The experience could be replicated in UNIX shells, but noone sems to care to actually go down that path.

TeMPOraL 6y ago
I feel that the closest we can get to a Lisp machine with modern components right now would be to get PowerShell, Emacs and StumpWM (or EXWM) to deeply interoperate.
ticviking 6y ago
I mean it's possible to run PowerShell on Linux, but PowerShell has some pretty significant rough edges, including a pretty painful performance story for something I want to use as a UI.
- pjmlp 6y ago
  It is also a matter of culture, as mentioned you could have a UNIX shell being structured, able to directly interact with shared objects, IPC with running applications, yet very few do it.

marlowe221 6y ago

I work in a Windows shop (security compliance reasons for federal government contracts). Before I got this job I was pretty much only developing in Linux. Honestly? Powershell is pretty cool, though it feels a little strange to admit that. It has it's own very powerful syntax and virtually all of the common Bash commands are now aliased by default, so it's pretty smooth transition for the basic stuff.

richardwhiuk 6y ago
See https://github.com/PowerShell/PowerShell/pull/1901 https://github.com/PowerShell/PowerShell/pull/1901