6 ms·

You can install PowerShell on Linux now, if you like - or there's nushell: https://www.nushell.sh/ https://www.nushell.sh/

by dflock 4y ago

skissane 4y ago
Two big differences between PowerShell (even on Linux) and traditional Unix shells: (1) multithreaded single process architecture, as opposed to multi-process architecture; (2) runs under a VM (.Net CLR) as opposed to native code. I see nushell is written in Rust, so it doesn't have (2), although I'm less sure about (1). I think it would be cool if you could add structured pipeline support to traditional Unix tools (bash, coreutils, etc). The problem is, Unix pipes don't have any support for passing metadata (performing content negotiation between the two ends of the pipe). Given they are a facility provided by the kernel, adding that would likely require kernel changes. One day I started daydreaming about prototyping content negotiation support for Unix pipes. I was thinking of using CUSE (Linux character device driver in userspace) to do it. Never actually got around to trying though, other more pressing things to work on. Maybe some day I will get around with it, or maybe someone else will feel more enthusiastic about that than I do.
- hnlmorg 4y ago
  It's possible without any kernel changes. My shell (https://github.com/lmorg/murex https://github.com/lmorg/murex) already supports doing that. The way it works is it uses fd3 to communicate schema information so it can natively support all the existing "dumb" pipes without any modification but any new tools can be written to send objects instead (albeit byte encoded). It's not as elegant as PowerShell sending .NET objects natively, but then PowerShell doesn't work with existing CLI tools natively (it needs wrapper scripts to convert them into PowerShell commands). Whereas my shell is fully backwards compatible while still supporting a suite of additional functionality too.
  - skissane 4y ago
    Interesting idea. Although I worry about relying on fd3, it seems like it could be a bit fragile. An executable might have a bug in its file descriptor management which only gets triggered when it inherits an extra fd. The developers might never notice because they never run it under your shell, and then someday some user does. An idea I've had before: the shell should listen on a Unix domain socket, and put the path to that in an environment variable. Then commands could talk to the shell which invoked them. That could be used for this kind of content negotiation stuff. It could also be used to do other things – for example, a subprocess could modify the current directory of the user's shell, or its environment – there are scenarios in which being able to do that would be useful. A Unix domain socket path in an environment variable seems less likely to cause unexpected issues than inheriting an extra file descriptor.
    - hnlmorg 4y ago
      Yeah that's another possible approach. I did consider that myself but I didn't want to manage sockets in addition to processes, plus file descriptors would be idiomatic to pipelines. Another issue was that if a child process then spawns another process, the environmental variables might also be carried over, so how does that grandchild process get restricted from writing to the socket? (it is a fixable problem but fd's solved that problem a little easier). Please don't take this as a dismissive comment though. I'm definitely not suggesting that your method has more problems nor that my method is better. There is a lot I do like about your suggestion and it might be an option I pivot to if/when I do run into issues with fd3.
      - skissane 4y ago
        Not trying to be dismissive of your approach either. While I can think of ways it might cause problems in theory, I really have no idea how likely those problems would be in practice. > I did consider that myself but I didn't want to manage sockets in addition to processes, plus file descriptors would be idiomatic to pipelines. A socket is just a file descriptor at the end of the day. Some shells (I think some variants of ksh?) have actually used socketpair() to build pipelines instead of pipe(). > the environmental variables might also be carried over, so how does that grandchild process get restricted from writing to the socket? Unix domain sockets let you know the PID/UID/GID at the other end (SO_PEERCRED, SCM_CREDENTIALS, etc). So when the shell gets a connection, it can tell whether it is coming from one of its direct children, or a grandchild. However, I'm not sure if one actually should restrict subprocess. If a child process spawns a subprocess, why shouldn't the subprocess be allowed to negotiate the input/output format, etc? The parent process might just be some kind of launcher and the child might be the one that does all the actual work. In any event, what's to stop fd3 being inherited by grandchild processes too?
- pjmlp 4y ago
  .NET CLR compiles to native code via JIT, and PowerShell also has support for calling into COM and DLL libraries. Additionally the way Xerox and ETHZ workstations did their REPLs was native code all the way, as Interlisp, Smalltalk, Mesa XDE, Mesa/Cedar, Oberon, Oberon-2, AOS all had AOT/JIT workflows.

You can install PowerShell on Linux now, if you like - or there's nushell: https://www.nushell.sh/ https://www.nushell.sh/

by dflock 4y ago

You can install PowerShell on Linux now, if you like - or there's nushell: https://www.nushell.sh/ https://www.nushell.sh/

skissane 4y ago

Two big differences between PowerShell (even on Linux) and traditional Unix shells: (1) multithreaded single process architecture, as opposed to multi-process architecture; (2) runs under a VM (.Net CLR) as opposed to native code. I see nushell is written in Rust, so it doesn't have (2), although I'm less sure about (1). I think it would be cool if you could add structured pipeline support to traditional Unix tools (bash, coreutils, etc). The problem is, Unix pipes don't have any support for passing metadata (performing content negotiation between the two ends of the pipe). Given they are a facility provided by the kernel, adding that would likely require kernel changes. One day I started daydreaming about prototyping content negotiation support for Unix pipes. I was thinking of using CUSE (Linux character device driver in userspace) to do it. Never actually got around to trying though, other more pressing things to work on. Maybe some day I will get around with it, or maybe someone else will feel more enthusiastic about that than I do.

hnlmorg 4y ago
It's possible without any kernel changes. My shell (https://github.com/lmorg/murex https://github.com/lmorg/murex) already supports doing that. The way it works is it uses fd3 to communicate schema information so it can natively support all the existing "dumb" pipes without any modification but any new tools can be written to send objects instead (albeit byte encoded). It's not as elegant as PowerShell sending .NET objects natively, but then PowerShell doesn't work with existing CLI tools natively (it needs wrapper scripts to convert them into PowerShell commands). Whereas my shell is fully backwards compatible while still supporting a suite of additional functionality too.
- skissane 4y ago
  Interesting idea. Although I worry about relying on fd3, it seems like it could be a bit fragile. An executable might have a bug in its file descriptor management which only gets triggered when it inherits an extra fd. The developers might never notice because they never run it under your shell, and then someday some user does. An idea I've had before: the shell should listen on a Unix domain socket, and put the path to that in an environment variable. Then commands could talk to the shell which invoked them. That could be used for this kind of content negotiation stuff. It could also be used to do other things – for example, a subprocess could modify the current directory of the user's shell, or its environment – there are scenarios in which being able to do that would be useful. A Unix domain socket path in an environment variable seems less likely to cause unexpected issues than inheriting an extra file descriptor.
  - hnlmorg 4y ago
    Yeah that's another possible approach. I did consider that myself but I didn't want to manage sockets in addition to processes, plus file descriptors would be idiomatic to pipelines. Another issue was that if a child process then spawns another process, the environmental variables might also be carried over, so how does that grandchild process get restricted from writing to the socket? (it is a fixable problem but fd's solved that problem a little easier). Please don't take this as a dismissive comment though. I'm definitely not suggesting that your method has more problems nor that my method is better. There is a lot I do like about your suggestion and it might be an option I pivot to if/when I do run into issues with fd3.
    - skissane 4y ago
      Not trying to be dismissive of your approach either. While I can think of ways it might cause problems in theory, I really have no idea how likely those problems would be in practice. > I did consider that myself but I didn't want to manage sockets in addition to processes, plus file descriptors would be idiomatic to pipelines. A socket is just a file descriptor at the end of the day. Some shells (I think some variants of ksh?) have actually used socketpair() to build pipelines instead of pipe(). > the environmental variables might also be carried over, so how does that grandchild process get restricted from writing to the socket? Unix domain sockets let you know the PID/UID/GID at the other end (SO_PEERCRED, SCM_CREDENTIALS, etc). So when the shell gets a connection, it can tell whether it is coming from one of its direct children, or a grandchild. However, I'm not sure if one actually should restrict subprocess. If a child process spawns a subprocess, why shouldn't the subprocess be allowed to negotiate the input/output format, etc? The parent process might just be some kind of launcher and the child might be the one that does all the actual work. In any event, what's to stop fd3 being inherited by grandchild processes too?
pjmlp 4y ago
.NET CLR compiles to native code via JIT, and PowerShell also has support for calling into COM and DLL libraries. Additionally the way Xerox and ETHZ workstations did their REPLs was native code all the way, as Interlisp, Smalltalk, Mesa XDE, Mesa/Cedar, Oberon, Oberon-2, AOS all had AOT/JIT workflows.