4 ms·

I think Powershell tried to do something like this article suggests. I appreciate the efforts, but do agree with the common criticism of verbosity. Also, it go

I think Powershell tried to do something like this article suggests. I appreciate the efforts, but do agree with the common criticism of verbosity. Also, it goes to show that no matter how bad aspects of bash are, the power accumulated over the years is hard to match, and a lot of that power is strongly coupled to the arcane and terse conventions.

heavenlyblue 7y ago
I think we may be on the edge of that transformative moment though. I had spent 2 hours yesterday trying to sort CSV lines in bash. Decided to do so even though I have a nice strongly typed framework to manage tuples, which is slower than bash. I always forget that the -k argument requires 2 numbers if you want to sort by only one column. Then LC_ALL. These are not the things that are so easy to debug against. So in the end it would have been faster to use Python. There’s xsv-rs, but it doesn’t provide the same semantics or power I have in the framework. Point is, I think there’s some space for novelty in UNIX tools. Especially given the research in DSLs for defining the transformations of tuples.
- lapnitnelav 7y ago
  You ever considered using CSVKit? It does a good job at being a subset of shell tools tailored to structured data as CSV/TSV and such.
  - heavenlyblue 7y ago
    Sad thing is, I am not sure at this point if my framework is more powerful than this or not. I think mine preserves the semantics of “sql execution plan” better than CSVKit, but I might as well be wrong. I did not know about CSVKit three years ago; but I don’t think it would take a prof. developer to build it too much time either.
6thaccount2 7y ago
Powershell is almost good, yet it falls off the mark in about a dozen ways. A lot of very simple things are harder to do (text is easier than objects up until a point) since Powershell treats everything as objects and LinuxCLI/Bash does everything as text being piped around to specific commands. The biggest complaint I have with Powershell is that it is so obnoxiously slow at parsing text files, that it just can't do basic scripting tasks on even medium sized files ~100MB without tons of StreamReader[] overhead or without calling a pre-compiled C# binary directly. This is the biggest fail. Nearly all the obvious ways of appending to files or parsing them stop working once the file size gets bigger than tiny. This isn't a problem with Linux commands that just pipe text to highly optimized C binaries. I'd drop Python for 90% of my scripting tasks on Windows if Powershell were faster in this area, but it is just painfully slow (if all the obvious ways of doing something in Powershell take ~5min when even Python can do it in ~4sec, you have a problem). However, you can usually write some very beautiful and easy to read code by using the built-in objects once you learn them. I also like how powershell's functions allow you to create commands with parameters in a super easy way so you can easily build DSLs in a way. Everyone feels differently, but calling my commands in this manner (InvokeCommand -param1 "blah") reads a lot more naturally to me than (Object.Method(param1)). Also, Powershell can handle things like filesystem interaction in a way that doesn't feel like a bolted on afterthought as Python's does.
- vnorilo 7y ago
  Thank you for the insight. I don't do a lot of powershell daily, but the last time I had a big batch job I was disappointed in how clunky it was to make a foreach-object -like pipeline utilize a sensible number of processes. I have forgotten the details but I think I needed some sort for add-on that split jobs across processes. Also, the pipeline wasn't pipelined in the parallelism-sense.
  - 6thaccount2 7y ago
    No problem. I'm certainly not an expert, but that is mainly due to the deficiency I mention above. I'd use it for a lot more use cases if it could handle those scenarios. I think Powershell "jobs" can help run things in parallel, but I've never done more than just read about it. The Powershell in Action book (3rd edition) covers that functionality I think. The first chapter to that book is pretty awesome. My dream as someone forced to use Windows would be for Microsoft to use some dollars to make Powershell a one stop shop for data science, simple games (think commodore sprites), numerical computing...etc. That sounds a little wacky I know, but I think people would appreciate every computer shipping with a powerful environment like that. Since Powershell is built on .NET, surely someone could make a library for Powershell that abstracts away some of the .NET complexity to where I can just do something like: Process-Data -data "blah.csv" | Create-PieChart -output "chart.png" Not having to install Anaconda and import a ton of libraries on the target computer would be helpful. Note that I'm not saying to reproduce all of Matlab in Powershell, but putting some of the most popular routines as built-in commands would be pretty awesome.
mixmastamyk 7y ago
> no matter how bad aspects of bash are, the power accumulated over the years is hard to match. I believe fish + python have a higher power/WTF ratio than bash and work in the same space. Perl is mentioned as well, but has fallen out of favor.

heavenlyblue 7y ago

I think we may be on the edge of that transformative moment though. I had spent 2 hours yesterday trying to sort CSV lines in bash. Decided to do so even though I have a nice strongly typed framework to manage tuples, which is slower than bash. I always forget that the -k argument requires 2 numbers if you want to sort by only one column. Then LC_ALL. These are not the things that are so easy to debug against. So in the end it would have been faster to use Python. There’s xsv-rs, but it doesn’t provide the same semantics or power I have in the framework. Point is, I think there’s some space for novelty in UNIX tools. Especially given the research in DSLs for defining the transformations of tuples.

lapnitnelav 7y ago
You ever considered using CSVKit? It does a good job at being a subset of shell tools tailored to structured data as CSV/TSV and such.
- heavenlyblue 7y ago
  Sad thing is, I am not sure at this point if my framework is more powerful than this or not. I think mine preserves the semantics of “sql execution plan” better than CSVKit, but I might as well be wrong. I did not know about CSVKit three years ago; but I don’t think it would take a prof. developer to build it too much time either.

6thaccount2 7y ago

Powershell is almost good, yet it falls off the mark in about a dozen ways. A lot of very simple things are harder to do (text is easier than objects up until a point) since Powershell treats everything as objects and LinuxCLI/Bash does everything as text being piped around to specific commands. The biggest complaint I have with Powershell is that it is so obnoxiously slow at parsing text files, that it just can't do basic scripting tasks on even medium sized files ~100MB without tons of StreamReader[] overhead or without calling a pre-compiled C# binary directly. This is the biggest fail. Nearly all the obvious ways of appending to files or parsing them stop working once the file size gets bigger than tiny. This isn't a problem with Linux commands that just pipe text to highly optimized C binaries. I'd drop Python for 90% of my scripting tasks on Windows if Powershell were faster in this area, but it is just painfully slow (if all the obvious ways of doing something in Powershell take ~5min when even Python can do it in ~4sec, you have a problem). However, you can usually write some very beautiful and easy to read code by using the built-in objects once you learn them. I also like how powershell's functions allow you to create commands with parameters in a super easy way so you can easily build DSLs in a way. Everyone feels differently, but calling my commands in this manner (InvokeCommand -param1 "blah") reads a lot more naturally to me than (Object.Method(param1)). Also, Powershell can handle things like filesystem interaction in a way that doesn't feel like a bolted on afterthought as Python's does.

vnorilo 7y ago
Thank you for the insight. I don't do a lot of powershell daily, but the last time I had a big batch job I was disappointed in how clunky it was to make a foreach-object -like pipeline utilize a sensible number of processes. I have forgotten the details but I think I needed some sort for add-on that split jobs across processes. Also, the pipeline wasn't pipelined in the parallelism-sense.
- 6thaccount2 7y ago
  No problem. I'm certainly not an expert, but that is mainly due to the deficiency I mention above. I'd use it for a lot more use cases if it could handle those scenarios. I think Powershell "jobs" can help run things in parallel, but I've never done more than just read about it. The Powershell in Action book (3rd edition) covers that functionality I think. The first chapter to that book is pretty awesome. My dream as someone forced to use Windows would be for Microsoft to use some dollars to make Powershell a one stop shop for data science, simple games (think commodore sprites), numerical computing...etc. That sounds a little wacky I know, but I think people would appreciate every computer shipping with a powerful environment like that. Since Powershell is built on .NET, surely someone could make a library for Powershell that abstracts away some of the .NET complexity to where I can just do something like: Process-Data -data "blah.csv" | Create-PieChart -output "chart.png" Not having to install Anaconda and import a ton of libraries on the target computer would be helpful. Note that I'm not saying to reproduce all of Matlab in Powershell, but putting some of the most popular routines as built-in commands would be pretty awesome.

mixmastamyk 7y ago

> no matter how bad aspects of bash are, the power accumulated over the years is hard to match. I believe fish + python have a higher power/WTF ratio than bash and work in the same space. Perl is mentioned as well, but has fallen out of favor.