5 ms·

Even on modern hardware with vectorization, the speed of perl is hard to match for stream processing. I write and deploy brand new perl code in 2022 to replace

by csdvrx 4y ago

Even on modern hardware with vectorization, the speed of perl is hard to match for stream processing. I write and deploy brand new perl code in 2022 to replace some clunky python and javascript - for real! AMA!

clscott 4y ago
1. What's your top technique for keeping it fast? 2. Are you hiring?
- csdvrx 4y ago
  > 1. What's your top technique for keeping it fast? Using the stock install as much as possible. Since a perl interpreter is available about everywhere, this makes the scripts cross platform very easily too. > 2. Are you hiring? Maybe? You have no email in your profile. Can you please send your CV to my nickname at outlook dot com?
wistlo 4y ago
I'm working in Windows (employer's choice) and need to process some huge data files on the desktop. Perl (Strawberry Perl for Windows) is much faster than native Powershell (or CMD) scripts. A few Perl one-liners and I have 2 GB processed in a couple of minutes. Big fan of Perl here.
- anthk 4y ago
  Perl is the Unix utilities' replacement for Windows. It can do awk/sed in one place. What I miss it's a "strings(1)" replacement for Perl.
  - kal247 4y ago
    There's a "strings" implementation in the PerlPowerTools : https://metacpan.org/pod/PerlPowerTools https://metacpan.org/pod/PerlPowerTools
    - anthk 4y ago
      TIL. Good addon for w32 perl users.
- csdvrx 4y ago
  > A few Perl one-liners and I have 2 GB processed in a couple of minutes. This matches my experience: my task is IO bound, processing tens of GB in a matter of minutes. > I'm working in Windows (employer's choice) Work prefers MacOS, Windows is my personal choice :) Windows 10 and 11 are a pleasure to use: between the Windows Terminal and tools like AutoHotKey, there's no Linux equivalent, even for a commandline geek. Using Windows (and Perl, and other weird things I like) may not be fashionable, but it's hard to beat.
  - wistlo 4y ago
    Full disclosure: Work has been Windows centric, but I recently moved to a new group with some Mac users. I'll have that choice, next time I swap hardware, but probably will stay with Windows because * Would miss too many amazingly useful utilities like arsclip. * Easier to patch into my home network, which is NTFS-based. * Employer is now wedded to Microsoft and Azure, so having Powershell and native access to NTFS AD on my native desktop is proving more useful. * I actually like the Windows interface, even with the Control Panel and other system utilities caught in a weird split between Windows XP and WIndows 10 UIs. * Some things, like certificate management and hosting a personal database, I do in Ubuntu or Debian as a VirtualBox guest under Windows. * Wife uses Windows on her laptop. She doesn't know it, but it's actually running as a VM on Debian, very stable, and can easily take her Windows to a new machine. * [I keep coming back to add items to this list] Remote Desktop Services have improved vastly, and I can manage remote servers or 12 year old client machines at a remote site with equal ease. (XP/7-era machines got a second life when I converted Windows OS disk to SSD). Also find screen sharing via 3rd party VNC (TightVNC) useful. * The keystrokes have become second-nature. Try as I might, I can never get fully comfortable on a Mac OS desktop. Now get off my lawn.
    - csdvrx 4y ago
      > The keystrokes have become second-nature. Try as I might, I can never get fully comfortable on a Mac OS desktop. Same, that's one of the biggest reasons - but I also agree with the rest of your list: what I like the most in Windows is the UI! I run most of my programs fullscreen, with shortcuts. I rarely use the mouse, and Windows plays along, while I have to fight other systems. Even when run baremetal, Windows has improved so much over the board that once you add RDP/VNC, it's a no brainer.
- bluedino 4y ago
  I don't think you could be slower than Powershell if you tried.
qsort 4y ago
What's the difference when compared with other scripting languages? Is it just all-around faster or it's about some particular application?
- csdvrx 4y ago
  > What's the difference when compared with other scripting languages? Compared to every other scripted languages I've tried, it's always faster - so much that it often rivals compiled code (even with extra optimizations as explained before) > Is it just all-around faster Yes - also faster to deploy, and harder to break in deployment due to dependencies rarely evolving. No need to pip install whatever. Consequently, tracking modules' versions (say in python) and their break in compatibility is a thing of the past. Some may say it's because it's dead. I guess I'm a necrophiliac then, because it makes my life so much easier that I'm loving it :)
avyjit 4y ago
When you refer to modern hardware with vectorization, are you saying perl beats software that makes use of vector instructions? If so wow! Afaik perl is a plain old interpreted language with no JIT, what makes it so fast? I had an idea of perl as in the same performance category as Python, Ruby and friends.
- csdvrx 4y ago
  > Afaik perl is a plain old interpreted language with no JIT, what makes it so fast? IDK. I didn't waste time checking why. My guess is that due to the much smaller code size, it reaches a sweetpoint where it fits nicely in the L1 or L2 cache. But it's just a hunch, I couldn't prove it.
- kstrauser 4y ago
  Perl likely spends a lot more time in C than a lot of other bytecode-compiled languages like Python. while(<STDIN>) { print $_; } isn't doing much of its work at the top level.
- cogman10 4y ago
  My assumption (and I could be wrong) is that whenever you do something that's ultimately CPU intense, perl dives into C code. The sort of stuff you do with perl tends to be a lot of string manipulation and it turns out perl is pretty hyper optimized for that sort of workload. Other languages could do the same, but often they have a few more layers before they start running that C code. I'd be curious to know if the perl grammar also somehow lends itself to being fairly optimizable for it's interpreter.
- jandrese 4y ago
  Perl does a kind of half JIT when you start a script. It doesn't compile fully down to native machine code, but it does parse and lex the script to build an internal representation. The code is then interpreted, but since it doesn't have to parse each line again it runs very fast and also you can interpret code on the fly in a variable so you get the best of both worlds.
  - 4y ago
auraham 4y ago
Any chance to make it open source on GitHub?

by csdvrx 4y ago

clscott 4y ago

1. What's your top technique for keeping it fast? 2. Are you hiring?

csdvrx 4y ago
> 1. What's your top technique for keeping it fast? Using the stock install as much as possible. Since a perl interpreter is available about everywhere, this makes the scripts cross platform very easily too. > 2. Are you hiring? Maybe? You have no email in your profile. Can you please send your CV to my nickname at outlook dot com?

wistlo 4y ago

I'm working in Windows (employer's choice) and need to process some huge data files on the desktop. Perl (Strawberry Perl for Windows) is much faster than native Powershell (or CMD) scripts. A few Perl one-liners and I have 2 GB processed in a couple of minutes. Big fan of Perl here.

anthk 4y ago
Perl is the Unix utilities' replacement for Windows. It can do awk/sed in one place. What I miss it's a "strings(1)" replacement for Perl.
- kal247 4y ago
  There's a "strings" implementation in the PerlPowerTools : https://metacpan.org/pod/PerlPowerTools https://metacpan.org/pod/PerlPowerTools
  - anthk 4y ago
    TIL. Good addon for w32 perl users.
csdvrx 4y ago
> A few Perl one-liners and I have 2 GB processed in a couple of minutes. This matches my experience: my task is IO bound, processing tens of GB in a matter of minutes. > I'm working in Windows (employer's choice) Work prefers MacOS, Windows is my personal choice :) Windows 10 and 11 are a pleasure to use: between the Windows Terminal and tools like AutoHotKey, there's no Linux equivalent, even for a commandline geek. Using Windows (and Perl, and other weird things I like) may not be fashionable, but it's hard to beat.
- wistlo 4y ago
  Full disclosure: Work has been Windows centric, but I recently moved to a new group with some Mac users. I'll have that choice, next time I swap hardware, but probably will stay with Windows because * Would miss too many amazingly useful utilities like arsclip. * Easier to patch into my home network, which is NTFS-based. * Employer is now wedded to Microsoft and Azure, so having Powershell and native access to NTFS AD on my native desktop is proving more useful. * I actually like the Windows interface, even with the Control Panel and other system utilities caught in a weird split between Windows XP and WIndows 10 UIs. * Some things, like certificate management and hosting a personal database, I do in Ubuntu or Debian as a VirtualBox guest under Windows. * Wife uses Windows on her laptop. She doesn't know it, but it's actually running as a VM on Debian, very stable, and can easily take her Windows to a new machine. * [I keep coming back to add items to this list] Remote Desktop Services have improved vastly, and I can manage remote servers or 12 year old client machines at a remote site with equal ease. (XP/7-era machines got a second life when I converted Windows OS disk to SSD). Also find screen sharing via 3rd party VNC (TightVNC) useful. * The keystrokes have become second-nature. Try as I might, I can never get fully comfortable on a Mac OS desktop. Now get off my lawn.
  - csdvrx 4y ago
    > The keystrokes have become second-nature. Try as I might, I can never get fully comfortable on a Mac OS desktop. Same, that's one of the biggest reasons - but I also agree with the rest of your list: what I like the most in Windows is the UI! I run most of my programs fullscreen, with shortcuts. I rarely use the mouse, and Windows plays along, while I have to fight other systems. Even when run baremetal, Windows has improved so much over the board that once you add RDP/VNC, it's a no brainer.
bluedino 4y ago
I don't think you could be slower than Powershell if you tried.

qsort 4y ago

What's the difference when compared with other scripting languages? Is it just all-around faster or it's about some particular application?

csdvrx 4y ago
> What's the difference when compared with other scripting languages? Compared to every other scripted languages I've tried, it's always faster - so much that it often rivals compiled code (even with extra optimizations as explained before) > Is it just all-around faster Yes - also faster to deploy, and harder to break in deployment due to dependencies rarely evolving. No need to pip install whatever. Consequently, tracking modules' versions (say in python) and their break in compatibility is a thing of the past. Some may say it's because it's dead. I guess I'm a necrophiliac then, because it makes my life so much easier that I'm loving it :)

avyjit 4y ago

When you refer to modern hardware with vectorization, are you saying perl beats software that makes use of vector instructions? If so wow! Afaik perl is a plain old interpreted language with no JIT, what makes it so fast? I had an idea of perl as in the same performance category as Python, Ruby and friends.

csdvrx 4y ago
> Afaik perl is a plain old interpreted language with no JIT, what makes it so fast? IDK. I didn't waste time checking why. My guess is that due to the much smaller code size, it reaches a sweetpoint where it fits nicely in the L1 or L2 cache. But it's just a hunch, I couldn't prove it.
kstrauser 4y ago
Perl likely spends a lot more time in C than a lot of other bytecode-compiled languages like Python. while(<STDIN>) { print $_; } isn't doing much of its work at the top level.
cogman10 4y ago
My assumption (and I could be wrong) is that whenever you do something that's ultimately CPU intense, perl dives into C code. The sort of stuff you do with perl tends to be a lot of string manipulation and it turns out perl is pretty hyper optimized for that sort of workload. Other languages could do the same, but often they have a few more layers before they start running that C code. I'd be curious to know if the perl grammar also somehow lends itself to being fairly optimizable for it's interpreter.
jandrese 4y ago
Perl does a kind of half JIT when you start a script. It doesn't compile fully down to native machine code, but it does parse and lex the script to build an internal representation. The code is then interpreted, but since it doesn't have to parse each line again it runs very fast and also you can interpret code on the fly in a variable so you get the best of both worlds.
- 4y ago

auraham 4y ago

Any chance to make it open source on GitHub?