8 ms·

This is pretty impressive. Powershell has been pretty slow for a lot of my uses lately. The author said this took ~30 secs. How long would a similar script take

This is pretty impressive. Powershell has been pretty slow for a lot of my uses lately. The author said this took ~30 secs. How long would a similar script take in Python? They're both slow on the language performance spectrum, but I bet Python would be significantly faster if anyone is less lazy than me to actually write some code.

marviel 9y ago
Seems a good place as any to mention the superb Trimesh library in python[1], which makes working with and creating .stl and .obj files very easy. [1]https://github.com/mikedh/trimesh https://github.com/mikedh/trimesh
na85 9y ago
On my old i7 powered ThinkPad, generating flow data from a 2D airfoil modeled using vortex panels was nearly instantaneous using Python. Can't imagine that generating 40 vertices would be taxing for it.
mpw222 9y ago
PowerShell is terribly slow compared to basically anything, but the slow bit here is probably the I/O. Each Add-Content call is a opening appending and flushing to the file, in the inner loop. The fact that Foreach-Object is vastly slower than the foreach keyword and that Add-Content itself is much slower than System.IO doesn't help, but I suspect this is basically I/O and the fact that this type of file access pattern tends to drive AV software crazy. Using a StreamWriter and getting the associated buffering would probably be a lot faster.
da_chicken 9y ago
PowerShell is slow, but writing to files the way he is doing is just a bad pattern. He uses Add-Content not just in loops, but in nested loops. The problem with that is that each time Add-Content is called the system opens the file, adds the content, flushes to disk, and then closes the file. That's a lot of overhead. In his wine glass script he also uses the PowerShell patten of creating an array ($x = @()) and then appending to it (foreach ($i in $set) { $x += $value * $i }). The problem here is that PowerShell arrays are fixed in size. To append a value to an array, the system creates a new array, copies all the values over with the new one, and then disposes of the old array. It works fine up to about 100 items, but it gets noticeably slow after that. Since it's done with a loop it makes it somewhere between O(n log n) and O(n^2). It's better to just output all the values as a single array ($x = $set | ForEach { $value * $_ }) or to create an ArrayList or List<String> or some C# collection that supports an O(1) append. He also assigns a lot of variables in his loops and then uses them only once to format a string. He could eliminate those variables and just embed the expressions in the strings. I got a ~25% performance improvement (210 ms to under 140 ms) with just the script embedded in the article when I switched to StringBuilder instead of Add-Content, and that script doesn't have the poor array pattern. StreamWriter would work, too, with less memory pressure than a StringBuilder. I suspect that with better code you could easily get this down below 3 seconds.
- lburton 9y ago
  I got a lot more than a 25% improvement.. you seem to have a much quicker machine or perhaps you were running one of the earlier scripts? For the wine glass script it went from >2 minutes to 1.5 seconds when output buffered. https://github.com/cosmosdarwin/obj-in-powershell/pull/1 https://github.com/cosmosdarwin/obj-in-powershell/pull/1 Also for those that haven't come across it https://github.com/dlwyatt/PowershellProfiler https://github.com/dlwyatt/PowershellProfiler is pretty useful.. despite add-content being the obvious offender here..
  - da_chicken 9y ago
    I have a 10 year old PC. The 210ms to 140 ms time was with the single cylinder script, which is much shorter, and the < 3 second prediction was for the wine glass script. I just guessed by looking that it wouldn't be hard to improve by an order of magnitude. Sorry if I wasn't clear!
  - juststeve 9y ago
    get-content is very slow as well
- throwaway7645 9y ago
  Thanks for the analysis. This is one of my main PS issues in that all the obvious ways are in fact wrong. I tried 3-4 different file write methods before and all were unbearably slow.
  - ygra 9y ago
    I wouldn't say any obvious way is wrong. Dropping down to the .NET mechanisms should be more of a last resort if performance is really abysmal for some reason, but apart from that, common sense applies. As they said, opening, writing to, and closing a file repeatedly in a loop is a stupid idea. I wouldn't say that's an obvious way to do file I/O in PowerShell. On that note, what are the obvious ways you've tried?
    - 9y ago

marviel 9y ago

Seems a good place as any to mention the superb Trimesh library in python[1], which makes working with and creating .stl and .obj files very easy. [1]https://github.com/mikedh/trimesh https://github.com/mikedh/trimesh

na85 9y ago

On my old i7 powered ThinkPad, generating flow data from a 2D airfoil modeled using vortex panels was nearly instantaneous using Python. Can't imagine that generating 40 vertices would be taxing for it.

mpw222 9y ago

PowerShell is terribly slow compared to basically anything, but the slow bit here is probably the I/O. Each Add-Content call is a opening appending and flushing to the file, in the inner loop. The fact that Foreach-Object is vastly slower than the foreach keyword and that Add-Content itself is much slower than System.IO doesn't help, but I suspect this is basically I/O and the fact that this type of file access pattern tends to drive AV software crazy. Using a StreamWriter and getting the associated buffering would probably be a lot faster.

da_chicken 9y ago

PowerShell is slow, but writing to files the way he is doing is just a bad pattern. He uses Add-Content not just in loops, but in nested loops. The problem with that is that each time Add-Content is called the system opens the file, adds the content, flushes to disk, and then closes the file. That's a lot of overhead. In his wine glass script he also uses the PowerShell patten of creating an array ($x = @()) and then appending to it (foreach ($i in $set) { $x += $value * $i }). The problem here is that PowerShell arrays are fixed in size. To append a value to an array, the system creates a new array, copies all the values over with the new one, and then disposes of the old array. It works fine up to about 100 items, but it gets noticeably slow after that. Since it's done with a loop it makes it somewhere between O(n log n) and O(n^2). It's better to just output all the values as a single array ($x = $set | ForEach { $value * $_ }) or to create an ArrayList or List<String> or some C# collection that supports an O(1) append. He also assigns a lot of variables in his loops and then uses them only once to format a string. He could eliminate those variables and just embed the expressions in the strings. I got a ~25% performance improvement (210 ms to under 140 ms) with just the script embedded in the article when I switched to StringBuilder instead of Add-Content, and that script doesn't have the poor array pattern. StreamWriter would work, too, with less memory pressure than a StringBuilder. I suspect that with better code you could easily get this down below 3 seconds.

lburton 9y ago
I got a lot more than a 25% improvement.. you seem to have a much quicker machine or perhaps you were running one of the earlier scripts? For the wine glass script it went from >2 minutes to 1.5 seconds when output buffered. https://github.com/cosmosdarwin/obj-in-powershell/pull/1 https://github.com/cosmosdarwin/obj-in-powershell/pull/1 Also for those that haven't come across it https://github.com/dlwyatt/PowershellProfiler https://github.com/dlwyatt/PowershellProfiler is pretty useful.. despite add-content being the obvious offender here..
- da_chicken 9y ago
  I have a 10 year old PC. The 210ms to 140 ms time was with the single cylinder script, which is much shorter, and the < 3 second prediction was for the wine glass script. I just guessed by looking that it wouldn't be hard to improve by an order of magnitude. Sorry if I wasn't clear!
- juststeve 9y ago
  get-content is very slow as well
throwaway7645 9y ago
Thanks for the analysis. This is one of my main PS issues in that all the obvious ways are in fact wrong. I tried 3-4 different file write methods before and all were unbearably slow.
- ygra 9y ago
  I wouldn't say any obvious way is wrong. Dropping down to the .NET mechanisms should be more of a last resort if performance is really abysmal for some reason, but apart from that, common sense applies. As they said, opening, writing to, and closing a file repeatedly in a loop is a stupid idea. I wouldn't say that's an obvious way to do file I/O in PowerShell. On that note, what are the obvious ways you've tried?
  - 9y ago