5 ms·

My favorite thing about unix is its philosophy of writing programs that "do one thing and do it well"

by oimaz 12y ago

Someone1234 12y ago
I like that too, I just wish UNIX IPC pipes had more metadata than throwing raw strings at one another and hoping the receiving process understands. I know people are going to roll their eyes at this, but Powershell's IPC constructs are a step forward. Everything inherits from Object which has the following prototype: public class Object { public Object(); public string ToString(); public bool Equals(object obj); public static bool Equals(object objA, object objB); public static bool ReferenceEquals(object objA, object objB); public int GetHashCode(); public extern Type GetType(); } So if you want to just do strings you still can (string will inherit from type object, and must implement toString()). However if you want to do a List<Object> you still can, and better still the receiving process can use GetType() which has to be implemented, and then have a few different workflows to handle the type (e.g. List, string, et al). UNIX was created in 1969(!) so nobody can blame them for designing it the way they did. However if you were creating UNIX today you'd definitely want to look at OOP and inheritance as a core pillar. See this post for this topic: http://technet.microsoft.com/en-us/magazine/2007.07.powershell.aspx http://technet.microsoft.com/en-us/magazine/2007.07.powershe...
- Crito 12y ago
  I can write *nix command-line utilities with ease in any language that I please. Can the same be said for powershell utilities? My understanding is that you are pretty much restricted to using CLR languages. If we are accepting that sort of languae lock-in, I'd rather get something like scsh off the ground.
  - Arnavion 12y ago
    PowerShell utilities are implemented as CLR classes inheriting from a particular base class. As such, they must be written in a CLR language (or PowerShell itself, of course). Edit: Of course, you have the option of writing a minimal wrapper in PS that shells out to your other-language command but the interface between PS and non-PS programs is still based on a string commandline.
    - jude- 12y ago
      The other key limitation of PowerShell is that its commands all run in the same process (e.g. PowerShell is akin to a REPL). By contrast, UNIX programs each run in their own process, giving the programmer a much greater degree of freedom in the design and implementation of each program. You can do IPC with PowerShell [1], but it's nowhere near as simple as UNIX-style piping. [1] http://coders-corner.net/2014/05/11/inter-process-communication-with-named-pipes-part-01-transfer-a-byte-stream/ http://coders-corner.net/2014/05/11/inter-process-communicat...
      - Arnavion 12y ago
        PS has the concept of jobs, an amalgamation of code and environment that is serialized to a separate powershell.exe process (potentially even to a different networked Windows machine). Communicating with these remote commands is identical to communication with a command - piping in and out of objects.
  - Someone1234 12y ago
    That's a little circular. You can use any language you please because all languages support pipes as a concept and "understand" stdin/stdout/stderr, etc. There's nothing inherently linking the CRL to using objects within pipes. You could use for example JSON if you really wanted.
    - Crito 12y ago
      Reading/writing to/from files is a FAR lower bar than linking with the CRL. It's a lower bar than even using JSON over pipes; most languages will have good JSON libraries but none of them are as trivial as reading/writing lines to a file. I'm not opposed to lifting ourselves above text over unix pipes, but I think that if we are going to do it then we should go in for the penny, in for the pound. Such a system will be incompatible with existing unix tools, so we might as well drop all the technical debt. I don't think powershell is a large enough break from the traditional unix setup. Aside: I've noticed that there is an open source implementation of Powershell for Mono called `pash`. Does anybody use this?
  - Someone 12y ago
    That is not language lock-in, it is ABI lock-in, and that is exactly the same with Unix pipes. You can complain that the increase in ABI complexity isn't worth the extra features, but I don't think it is fair to complain about increased ABI complexity, period.
    - deleted 12y ago
      [deleted]
    - Crito 12y ago
      > "Reading/writing to/from files is a FAR lower bar than linking with the CRL."* If I want to write a unix utility in C, or Java, or C#, or Lua, or Racket, or postscript, or brainfuck..., I can. Not just in an academic sense, but in a "it is actually reasonable and trivial to do so" sense. The list of programming language implementations that cannot read/write to/from files (knowledge of the concept of "pipes" is unnecessary, the user's shell opens those) is vanishingly small. For the average developer who just wants to use Powershell/bash/whatever, it is absolutely language lock-in.
      - pjmlp 12y ago
        It is a common misconception that Powershell is CLR only. You can also make use of any COM, DCOM, or plain dynamic library in the shell. There are also text formatters for interoperability with older scripts.
- epistasis 12y ago
  Defaulting to a data serialization scheme like Avro or Protocol Buffers could be used here, especially with a small amount of shell support to automatically convert terminal-destined data serialization streams to a text string. It would be an interesting to see if any such projects exist, because it would fit very easily into the current pipe IPC infrastructure, with the addition of transformative pipelines for interfacing with the existing string-base utilities, but that's necessary anyway. I mean to say that there's no need to modify existing IPC mechanisms, just a need to establish a common serialization standard and embed it into a shell.
  - Arnavion 12y ago
    >Defaulting to a data serialization scheme like Avro or Protocol Buffers could be used here >I mean to say that there's no need to modify existing ICP mechanisms, just a need to establish a common serialization standard PS gives you two things though. It doesn't just give you objects, it also gives you the behaviors of those objects as implemented by the runtime (CLR) that is common between both endpoints of the communication. For example, consider a command that returns a list of processes running on the current session as a List of Process objects. This list is then piped to a command that filters the list to a sublist of processes that have a particular Name property. Finally, this sublist is passed to a third command that calls the Process.Kill() method on each of those objects. This requires that all three commands not only agree on the structure of a Process object (as protobuf would provide) but also that it has a Kill() method, etc. This is possible in PS because all three commands are running in the same common runtime, but not possible with generic IPC. IPC of complex objects has existed on Windows even before the CLR with COM and DCOM, and again both of these are more than serialization mechanisms, since they must support cross-process data as well as behavior.
    - epistasis 12y ago
      Interesting, in a unified object programming environment like the CLR I could see that being very useful. Such live methods could also be implemented in, say, Python by constructing objects from an Avro data stream. Or structs/stubs could be autopopluated in C with the proper library, but that would require the entire POSIX C world to be codified into a standard serialization. It would be a lot of work, but I think that it still fits inside an Avro-style data serialization scheme, plus proper runtime support. The runtime deserialization is really where the magic happens. And I think the serializaiton, deserialization is necessary to maintain proper process separation, particularly in languages like C.
- smorrow 12y ago
  I want to like this stuff, but I can't see it passing McIlroy's Test.
- jude- 12y ago
  > However if you were creating UNIX today you'd definitely want to look at OOP and inheritance as a core pillar. I doubt it. Imposing structure and types on IPC would make it very difficult to compose separate programs. Suppose I created UNIX-2, where the only difference between UNIX and UNIX-2 in principle is the fact that UNIX-2 programs all pipe serialized Objects to and from each other, instead of byte streams. Now, the ls program obviously outputs more information than just a list of strings--it also outputs types of files, permissions, owners, inode numbers, sizes, timestamps, etc. I might be inclined to have ls output an lsOutputObject (derived from Object) that encapsulated all this information. Suppose I wanted to pipe the output of ls into wc. How does wc handle an lsOutputObject? Either wc is programmed to know how to handle lsOutputObject, or it is not. Since we want an object-oriented environment, we'll assume the former case, so wc can call the appropriate object-specific methods and access the appropriate object-specific fields. But, now wc is tightly coupled to ls. This problem generalizes. For a given program P in a set of N programs, wc will need to know how to access P's output-object-specific fields and methods. So, wc needs O(N) different subroutines to interact with the N other programs. This does not scale--each additional program I add to UNIX-2 will require me to write O(N) additional IPC handlers--one for each program. The only way to avoid this IPC-handler-explosion in the design is to define the set of IPC objects a priori and mandate all programs know how to handle them. Then, there are O(1) IPC handlers per program, and adding a new program does not require me to couple its implementation to any other programs. This is effectively what UNIX does: there is one IPC object--a string of bytes. In UNIX-2 I could have more types of objects, but the fact that they're defined independent of the programs means that I will still be "hoping the receiving process understands" when I give it data from arbitrary programs. Suppose I relax this a priori object mandate above to allow programs to extend the base IPC object types. But then, programs that do so will only compose with programs that implement IPC handlers for their extended object types. In this scenario, I can expect there to be disjoint sets of programs that compose with one another, but not with others. This is effectively what happens in SOA/microservice architectures: you get a set of programs that are composible only with programs that speak their (arbitrarily-structured) messages (the set of which is much smaller than the global set of SOA/microservice programs). My point is, trying to enforce OOP on IPC will take away universal program composibility, which is the killer (if not defining) feature of UNIX.
- agumonkey 12y ago
  About 'IPC' that talk about Lisp Machines single namespace was interesting. Passing pointers instead of serialized data. www.youtube.com/watch?v=o4-YnLpLgtk
stinos 12y ago
"do one thing and do it well" which is basically the all-mighty-yet-way-too-often-violated SRP [1], with the context scaled up a bit from fuction/class/... to program [1] https://en.wikipedia.org/wiki/Single_responsibility_principle https://en.wikipedia.org/wiki/Single_responsibility_principl...

My favorite thing about unix is its philosophy of writing programs that "do one thing and do it well"

by oimaz 12y ago

My favorite thing about unix is its philosophy of writing programs that "do one thing and do it well"

I like that too, I just wish UNIX IPC pipes had more metadata than throwing raw strings at one another and hoping the receiving process understands. I know people are going to roll their eyes at this, but Powershell's IPC constructs are a step forward. Everything inherits from Object which has the following prototype: public class Object { public Object(); public string ToString(); public bool Equals(object obj); public static bool Equals(object objA, object objB); public static bool ReferenceEquals(object objA, object objB); public int GetHashCode(); public extern Type GetType(); } So if you want to just do strings you still can (string will inherit from type object, and must implement toString()). However if you want to do a List<Object> you still can, and better still the receiving process can use GetType() which has to be implemented, and then have a few different workflows to handle the type (e.g. List, string, et al). UNIX was created in 1969(!) so nobody can blame them for designing it the way they did. However if you were creating UNIX today you'd definitely want to look at OOP and inheritance as a core pillar. See this post for this topic: http://technet.microsoft.com/en-us/magazine/2007.07.powershell.aspx http://technet.microsoft.com/en-us/magazine/2007.07.powershe...

Crito 12y ago
I can write *nix command-line utilities with ease in any language that I please. Can the same be said for powershell utilities? My understanding is that you are pretty much restricted to using CLR languages. If we are accepting that sort of languae lock-in, I'd rather get something like scsh off the ground.
- Arnavion 12y ago
  PowerShell utilities are implemented as CLR classes inheriting from a particular base class. As such, they must be written in a CLR language (or PowerShell itself, of course). Edit: Of course, you have the option of writing a minimal wrapper in PS that shells out to your other-language command but the interface between PS and non-PS programs is still based on a string commandline.
  - jude- 12y ago
    The other key limitation of PowerShell is that its commands all run in the same process (e.g. PowerShell is akin to a REPL). By contrast, UNIX programs each run in their own process, giving the programmer a much greater degree of freedom in the design and implementation of each program. You can do IPC with PowerShell [1], but it's nowhere near as simple as UNIX-style piping. [1] http://coders-corner.net/2014/05/11/inter-process-communication-with-named-pipes-part-01-transfer-a-byte-stream/ http://coders-corner.net/2014/05/11/inter-process-communicat...
    - Arnavion 12y ago
      PS has the concept of jobs, an amalgamation of code and environment that is serialized to a separate powershell.exe process (potentially even to a different networked Windows machine). Communicating with these remote commands is identical to communication with a command - piping in and out of objects.
- Someone1234 12y ago
  That's a little circular. You can use any language you please because all languages support pipes as a concept and "understand" stdin/stdout/stderr, etc. There's nothing inherently linking the CRL to using objects within pipes. You could use for example JSON if you really wanted.
  - Crito 12y ago
    Reading/writing to/from files is a FAR lower bar than linking with the CRL. It's a lower bar than even using JSON over pipes; most languages will have good JSON libraries but none of them are as trivial as reading/writing lines to a file. I'm not opposed to lifting ourselves above text over unix pipes, but I think that if we are going to do it then we should go in for the penny, in for the pound. Such a system will be incompatible with existing unix tools, so we might as well drop all the technical debt. I don't think powershell is a large enough break from the traditional unix setup. Aside: I've noticed that there is an open source implementation of Powershell for Mono called `pash`. Does anybody use this?
- Someone 12y ago
  That is not language lock-in, it is ABI lock-in, and that is exactly the same with Unix pipes. You can complain that the increase in ABI complexity isn't worth the extra features, but I don't think it is fair to complain about increased ABI complexity, period.
  - deleted 12y ago
    [deleted]
  - Crito 12y ago
    > "Reading/writing to/from files is a FAR lower bar than linking with the CRL."* If I want to write a unix utility in C, or Java, or C#, or Lua, or Racket, or postscript, or brainfuck..., I can. Not just in an academic sense, but in a "it is actually reasonable and trivial to do so" sense. The list of programming language implementations that cannot read/write to/from files (knowledge of the concept of "pipes" is unnecessary, the user's shell opens those) is vanishingly small. For the average developer who just wants to use Powershell/bash/whatever, it is absolutely language lock-in.
    - pjmlp 12y ago
      It is a common misconception that Powershell is CLR only. You can also make use of any COM, DCOM, or plain dynamic library in the shell. There are also text formatters for interoperability with older scripts.
epistasis 12y ago
Defaulting to a data serialization scheme like Avro or Protocol Buffers could be used here, especially with a small amount of shell support to automatically convert terminal-destined data serialization streams to a text string. It would be an interesting to see if any such projects exist, because it would fit very easily into the current pipe IPC infrastructure, with the addition of transformative pipelines for interfacing with the existing string-base utilities, but that's necessary anyway. I mean to say that there's no need to modify existing IPC mechanisms, just a need to establish a common serialization standard and embed it into a shell.
- Arnavion 12y ago
  >Defaulting to a data serialization scheme like Avro or Protocol Buffers could be used here >I mean to say that there's no need to modify existing ICP mechanisms, just a need to establish a common serialization standard PS gives you two things though. It doesn't just give you objects, it also gives you the behaviors of those objects as implemented by the runtime (CLR) that is common between both endpoints of the communication. For example, consider a command that returns a list of processes running on the current session as a List of Process objects. This list is then piped to a command that filters the list to a sublist of processes that have a particular Name property. Finally, this sublist is passed to a third command that calls the Process.Kill() method on each of those objects. This requires that all three commands not only agree on the structure of a Process object (as protobuf would provide) but also that it has a Kill() method, etc. This is possible in PS because all three commands are running in the same common runtime, but not possible with generic IPC. IPC of complex objects has existed on Windows even before the CLR with COM and DCOM, and again both of these are more than serialization mechanisms, since they must support cross-process data as well as behavior.
  - epistasis 12y ago
    Interesting, in a unified object programming environment like the CLR I could see that being very useful. Such live methods could also be implemented in, say, Python by constructing objects from an Avro data stream. Or structs/stubs could be autopopluated in C with the proper library, but that would require the entire POSIX C world to be codified into a standard serialization. It would be a lot of work, but I think that it still fits inside an Avro-style data serialization scheme, plus proper runtime support. The runtime deserialization is really where the magic happens. And I think the serializaiton, deserialization is necessary to maintain proper process separation, particularly in languages like C.
smorrow 12y ago
I want to like this stuff, but I can't see it passing McIlroy's Test.
jude- 12y ago
> However if you were creating UNIX today you'd definitely want to look at OOP and inheritance as a core pillar. I doubt it. Imposing structure and types on IPC would make it very difficult to compose separate programs. Suppose I created UNIX-2, where the only difference between UNIX and UNIX-2 in principle is the fact that UNIX-2 programs all pipe serialized Objects to and from each other, instead of byte streams. Now, the ls program obviously outputs more information than just a list of strings--it also outputs types of files, permissions, owners, inode numbers, sizes, timestamps, etc. I might be inclined to have ls output an lsOutputObject (derived from Object) that encapsulated all this information. Suppose I wanted to pipe the output of ls into wc. How does wc handle an lsOutputObject? Either wc is programmed to know how to handle lsOutputObject, or it is not. Since we want an object-oriented environment, we'll assume the former case, so wc can call the appropriate object-specific methods and access the appropriate object-specific fields. But, now wc is tightly coupled to ls. This problem generalizes. For a given program P in a set of N programs, wc will need to know how to access P's output-object-specific fields and methods. So, wc needs O(N) different subroutines to interact with the N other programs. This does not scale--each additional program I add to UNIX-2 will require me to write O(N) additional IPC handlers--one for each program. The only way to avoid this IPC-handler-explosion in the design is to define the set of IPC objects a priori and mandate all programs know how to handle them. Then, there are O(1) IPC handlers per program, and adding a new program does not require me to couple its implementation to any other programs. This is effectively what UNIX does: there is one IPC object--a string of bytes. In UNIX-2 I could have more types of objects, but the fact that they're defined independent of the programs means that I will still be "hoping the receiving process understands" when I give it data from arbitrary programs. Suppose I relax this a priori object mandate above to allow programs to extend the base IPC object types. But then, programs that do so will only compose with programs that implement IPC handlers for their extended object types. In this scenario, I can expect there to be disjoint sets of programs that compose with one another, but not with others. This is effectively what happens in SOA/microservice architectures: you get a set of programs that are composible only with programs that speak their (arbitrarily-structured) messages (the set of which is much smaller than the global set of SOA/microservice programs). My point is, trying to enforce OOP on IPC will take away universal program composibility, which is the killer (if not defining) feature of UNIX.
agumonkey 12y ago
About 'IPC' that talk about Lisp Machines single namespace was interesting. Passing pointers instead of serialized data. www.youtube.com/watch?v=o4-YnLpLgtk

stinos 12y ago

"do one thing and do it well" which is basically the all-mighty-yet-way-too-often-violated SRP [1], with the context scaled up a bit from fuction/class/... to program [1] https://en.wikipedia.org/wiki/Single_responsibility_principle https://en.wikipedia.org/wiki/Single_responsibility_principl...