19 ms·
Perl first commit: a “replacement” for Awk and sed
- aportnoy 3y agoThe funny thing is Perl is now arguably more obsolete than sed and awk.
- jph00 3y agoThat might be true culturally, but it's not true technically. Once you know a bit of Perl (including some command line switches like -e, -n, -a, -l, -p, and -i) you will be able to do just as easily everything you used to do in awk and sed, but with a more powerful foundation so you have the flexibility to go further.
- imiric 3y ago> but with a more powerful foundation so you have the flexibility to go further. The question is: should you? I like Perl, but it itself has many warts that make maintaining a large codebase more of a nightmare than using C, and certainly more than most modern languages. I agree with other commenters here that the tools Perl sought to replace are still used more than it. It has its niche of being excellent at text processing, and more capable than shell scripts at that task, but I'd think twice about reaching for it to build anything more complex than a shell script replacement. Especially in 2023.
- whateveracct 3y agoWhat's a good resource to be proficient in perl for shell scripting? I write a lot of bash for better or worse. I wonder if perl would be a better choice sometimes.
- 878654Tom 3y agoIf you can't manage it with Shell scripting you should take a look at Python. I've written Perl for many years but in the end switched to Python because the internet and resources you can find are much more quantitative and qualitative than those for Perl. Especially now when Google doesn't seem to return older results anymore.
- cutler 3y agoWorse Is Better syndrome.
- whateveracct 3y agoI'd rather write perl tho
- cutler 3y ago"Minimal Perl" by Tim Maher is unrivaled for this.
- kagevf 3y agoThank you for mentioning that book - I'm very interested in how perl can replace awk and sed and even grep(!) - I use grep multiple times a day, awk maybe a few times a week, and sed very rarely. I think I'm in the target demographic for this :)
- version_five 3y agoMy first thought when I saw this was that I'd like to see some stats on use of the three. I personally use sed and awk regularly and never perl, but I have no idea what is typical. I have a feeling that most people (amongst those who may have a use case) don't use any of them.
- SoftTalker 3y agoYep I responded in another comment but I use sed and awk almost daily but I never learned Perl. It's actually something that could be useful in my current role though -- is there a particularly good intro to Perl in 2023 that I might want to look at?
- djbusby 3y agoThe wonderful thing about Perl is the docs from 2013 are still valid. All the growth/rapid-change/package-management issues that newer ecosystem suffer through Perl did in like the late 90s. 20 year old code still works (mostly). Not nearly the churn (anymore) as JS or PHP or Go or Rust.
- idlewords 3y agoThe Perl man pages are really good! Start with "man perlintro" and then look at whatever sections in the "man perl" table of contents fit your needs. A lot of the tutorials online in my opinion are overbaked and written for people writing large OO projects using a lot of scaffolding. But the man pages were originally targeted at sed and awk users in your position.
- SoftTalker 3y agoSounds perfect, thanks!
- explaininjs 3y agoGoogle Trends has sed crossing over perl about 5 years ago. Awk seems insignificant, which matches my expectation. (I’ve used sed but never awk, I don’t even know what it does) Worth noting “sed” is “thirst” in Spanish, which has the potential to throw off the data, especially worldwide. https://trends.google.com/trends/explore?date=all&geo=US&q=perl,awk,sed&hl=en-US https://trends.google.com/trends/explore?date=all&geo=US&q=p...
- shrubble 3y agoIt's still used heavily for internal processes and internal web sites that query databases etc. At least at some of the large publicly traded companies I know about...
- zeroonetwothree 3y agoI use Perl scripts dozens of times a day but I rarely write new code in it. awk and sed I use less often but it’s almost always ad hoc new scripts.
- pipeline_peak 3y agoUnix was always about tools doing one thing. Obviously awk and sed didn’t do one thing only, but Perl broke away from that entirely. Perl tried to replace awk, sed, and the shell all at once. The problem with Perl is it was still tied to Unix culture to compete with Python and it’s strong library set, so it was pretty much fazed out as an awkward intermediary.
- golergka 3y agoDoesn't booking.com still run on it?
- colejohnson66 3y agoAnd cPanel (used by many web hosts)?
- edgyquant 3y agoThats a name i haven’t heard in years
- smueller1234 3y agoYes, though not exclusively anymore.
- rawleyfowler 3y agoYes. Actually a lot of companies still run on it. Booking.com, Fastly, ByteMark, OpenCage, I even know of a few startups that run on Perl.
- idlewords 3y agoMy livelihood depends on a large set of Perl scripts, and I bet I'm not the only one in that position. It's perfect for gluing things together and then not breaking for decades, while other languages come and go. It's kind of the new COBOL that way.
- macintux 3y agoI’d love to eventually settle into retirement by maintaining some unloved Perl.
- djbusby 3y agoI make money off "old" systems; I'm spending time with Perl too. Lots of legacy systems out there.
- cutler 3y agoWhich languages are most common in these systems?
- djbusby 3y agoWell, I'm not sure about everywhere but, the work I'm finding is ASP3.0 (IIS6.0) and VB6 (more rare) also Perl and PHP. Many of these projects were started in the late 90s. They are internal systems (billing automation, internal work-tracking, custom "ERP"). Nothing exciting, just old shit that makes money in a non-tech business. I find the work on LI, and I show up when folk search for these olds skills. But I know a few others in this "maintenance" space. One just does legacy Java, one just does old Windows (NT4, 2000), etc. Generally we find work from each other. Additionally, I think it's hard to enter the "Legacy" space. You have to un-learn some patterns, lots of: oh yea, we used to do this the hard way. The other bump is the documentation - the 1999 docs are buried under the 2009 docs which are decaying under the 2019 docs. The thing called ActiveRecord for example means like 99 things.
- cutler 3y agoDo you find any of these legacy system clients are open to having their system upgraded to a modern version of the same or related language, eg. .Net for ASP? After all, it wouldn't involve a significant change in the business logic and most esablished languages are more or less backwards compatible.
- waynecochran 3y agoI use Perl all of the time. If I am going to write a bash script that is more than 12 lines long I use Perl or if I need to do read text files and do regex matching things I haven't found anything better.
- globular-toast 3y agoThere's a difference between using something because you want to and using something because you have to. awk and sed are sometimes the right tool, just not always. When I used sed and awk a lot I also made heavy use of bash which makes things a lot nicer. And Python, of course.
- windowshopping 3y agoDidn't git come out in 2005? Which version control was this using in 1987 that was converted into git compatibility?
- SoftTalker 3y agoProbably CVS or RCS.
- Wuzado 3y agoSending patches on mailing lists? Also TIL that patch (the UNIX program) was also created by Larry Wall.
- dan-robertson 3y agoI think someone synthesised a git history together from past versions, patches on usenet, etc
- autarch 3y agoIt was in Perforce before git. Not sure if there was something before Perforce.
- ForkMeOnTinder 3y agoI used to kind of unfairly disparage perl, but now I regret never learning it. I've written a lot of scripts for munging text/files that outgrow bash, and when I rewrite in Python they get twice as long. Seems like perl would be a happy medium and let me keep using a lot of the nice bash shortcuts like `baskticks` for shell execution, etc
- jph00 3y agoYeah so many people shooting themselves in the foot by failing to take advantage of this powerful tool - as you say especially for stuff that you might have otherwise used an overly-complex bash script for.
- nerdponx 3y agoI find Python much easier to read and write, whereas I find it hard to write Perl as anything but an unstructured blob. Its main advantage for me is that interpreter startup time is significantly faster than Python, Ruby, Node, and most Scheme implementations. So it's useful if I ever need to call an external program for some text processing inside a shell script. Otherwise I strongly prefer the maintainability of Python.
- quickthrower2 3y agoNow with LLMs to reduce feature discovery pain there is less of an excuse! Edit: this applies to any language not just Perl. I use it to learn Python things I would otherwise miss.
- SoftTalker 3y agoI once went looking for something like Ansible that was implemented in Perl. Does such a thing exist? I think declarative system orchestration got popular well after perl's popularity declined. In any case, I did not find anything.
- shrubble 3y agoThere is CFengine which can have modules for it written in Perl, but itself is not written in Perl.
- simcop2387 3y agoOne thing to keep in mind is that this is not actually the first version of perl, it's the first public version of perl. A version 0 did exist at the JPL previously but it's likely lost to the ages at this point. EDIT: For the curious about the history, take a look at the perlhist documentation, https://perldoc.perl.org/perlhist https://perldoc.perl.org/perlhist it's got a lot of good info about the history of perl and it's releases.
- rozab 3y agoThis doc has no info about perl version 0. Any source on it emerging at JPL? Can't find anything online except forum posts
- simcop2387 3y agoApparently my memory on that seems to be faulty, looks oike it was shortly after he was at jpl and left for unisys/system development corp (it changed names), https://www.oreilly.com/pub/au/148 https://www.oreilly.com/pub/au/148
- deleted 3y ago[deleted]
- avar 3y agoPerl was born as an NSA project, this old HN thread has some good references: https://news.ycombinator.com/item?id=14069320 https://news.ycombinator.com/item?id=14069320 As for as I know the only "official" reference to that is the "classified" and "don't ask" in perlhist.
- unpythonic 3y agoWhen perl came out we were living in horrific times. You had the choice of either Bourne, C or Korn shell. Automation was glued together in one of these with a series of grep, awk, sed, ls, test, commands glued together. Anything more complicated was written in C and called from one of these things. Perl in one stroke collapsed the programming of C, text manipulation, the capabilities of all of the Unix utilities, and data structures into one system. For anything which wasn't subsumed into the monolith of Perl, you could easy access via backticks. It was very friendly in dealing with text streams, and that's what those call-outs in those back ticks spoke. Yes, awk and sed were replaced by Perl, but more importantly, the unmaintainable nightmare that glued all of it together was wiped out.
- SoftTalker 3y agosed and awk are still around and still used a lot especially for one-liners. Anything more than that is typically not done as much anymore, though is certainly possible.
- LinuxBender 3y agosed and awk are still around and still used a lot especially for one-liners. Adding to that some popular examples are here [1][2] [1] - https://www.commandlinefu.com/commands/matching/awk/YXdr/sort-by-votes https://www.commandlinefu.com/commands/matching/awk/YXdr/sor... [2] - https://www.commandlinefu.com/commands/matching/sed/c2Vk/sort-by-votes https://www.commandlinefu.com/commands/matching/sed/c2Vk/sor...
- imron 3y ago> Yes, awk and sed were replaced by Perl, I still use awk and sed semi regularly. I haven’t used Perl in over a decade.
- idlewords 3y agoSounds like you were also replaced by Perl.
- 3y ago
- mulmen 3y agoI will always love perl because it got my career off the ground. Anyone who doesn’t just learn it is doing themselves a disservice. It’s similar to SQL in that sense.
- SoftTalker 3y agoPerl was huge in the early web and first dot-com boom, it was used a lot for CGI scripting.
- racl101 3y agoI probably didn't learn Perl because PHP superseded it for web stuff.
- alexjplant 3y agoThis. PHP was my first "real" programming language (which is a bit scary now that I think about it since it followed me learning QBasic and VB6) and by the time I was messing with it for web dev circa 2003 CGI scripts were on their way out. I suspect that it had something to do with CPanel/WHM-based shared hosting setups being reticent to deal with something as powerful as CGI and preferring to just run PHP in Safe Mode. I could be wrong though as I was also a technically-ignorant punk that fixed phpBB installer filesystem permission issues by 777'ing everything in my hosting directory :-). Good times...
- mulmen 3y agoHa! Cheers! My first language was VB.net and my first paying gig was porting a VB6 app. But my career really started when I took on Perl and Oracle. Good times indeed.
- mulmen 3y agoYou missed out. Perl is way more than the web. Nobody is grinding up csv files with PHP. But I wired up an Oracle billing system and a MSSQL invoice system in under a month as a junior dev. And it worked. Every library I needed was at my fingertips on CPAN. And just… worked. Connect to Oracle, run SQL, create a CSV, FTP it somewhere. EZPZ.
- _2z1p 3y agoUnpopular opinion, but I think we'd be so much better off if Netscape had just embedded a Perl interpreter instead of creating JavaScript. Perl CGI was already the dominant server side technology at the time and it has incredible string manipulation capabilities that would be beneficial for browser scripting.
- speedgoose 3y agoJavaScript being some kind of Lisp dialect with a C syntax is good IMHO. You also get some Perl inspired features in JavaScript, like the regular expression engine, and the not so user friendly array methods like slice and splice. The DOM API is also a bit more robust than string manipulation.
- jeremyjh 3y agoI know its history but Javascript is not in any meaningful sense a "Lisp dialect". Code is not data, code is not even an object, you have to define a function, closure or object to manipulate the code itself.
- runlaszlorun 3y agoI’m thinking Self would be a better reference but I’m guessing there’s a limited audience s who’s ever even seen it.
- runlaszlorun 3y agoI’m not sure ‘robust’ is a term that often comes to mind for me regarding the DOM API.
- akira2501 3y agoI recall the C interface to perl being exceptionally ugly. It was not designed with embedding in mind. That was one of the things that convinced me to move to ruby.. matz had exceptional taste when it came to the C api.
- casey2 3y ago
- racl101 3y agoWeird, I have never intentionally used Perl. Still use awk and sed.
- Scubabear68 3y agoI know people dump on Perl all the time, I know I do. But it was a critical part of computing history. Perl did a tremendous job of bridging the gap between shell commands and shell scripts and “big languages”. Way back in the early 90’s I took a complicated shell script that took about 2 hours to run this huge text processing job , rewrote it in Perl and it took about 20 seconds, and was more maintainable to boot. It really helped boost software development in the 90s, even pre-CGI script. The Perl source itself was also very interesting, with numerous optimizations to make it fast (again, for its time). What killed it were the never ending eccentricities you ran into, endless foot guns, and of course Perl 6 never being delivered. But it really was the weird shit that did it in.
- idlewords 3y agoDuring a later period (early 2000's) Perl was the only scripting language with good Unicode support, including inside regular expressions, which was revolutionary at the time. There was so much cool stuff you could only do in Perl, and then like you say it all ran into the brick wall of Perl 6.
- eduction 3y agoYes and Perl’s Unicode transition between 5.5 and 5.6* was exemplary. There were some speed bumps but generally old stuff continued to work and Unicode support could be added in a straightforward manner. In contrast to how python 3 handled it (not trying to start a language war, these days I prefer python overall). I suspect Larry Wall just understood text processing much more deeply than Guido (makes sense since he is a linguist and Perl was always more text oriented). *technically it was 5.005 or something to 5.6 because they decided to drop a zero or two
- btilly 3y agoThe brick wall of Perl 6 was itself a solution to an internal nightmare inside of core Perl development. Particularly around conflicts involving Tom Christensen, which caused no progress to be made on Perl 5.6.1 despite major bugs in 5.6.0. For example: my $x = 5 + 6; $x .= "0"; print $x + 5; # 16, not 115. The result in the year 2000 was a famous cup throwing incident by Jon Orwant. And Perl 6 was a plan to separate those who wanted to create their ideal language (Perl 6) from those who wanted to maintain existing Perl. And it worked, sort of, for a few years. I was on the Perl Grant Committee at the moment that killed Perl 6 in my opinion. We had a choice of 2 grants from Nicholas Clark that we could fund during 2006. One brought a lot of immediate benefits to Perl (Unicode fixes, reduced memory usage, etc), and the other was to get Ponie to the point where someone other than a core Perl developer could work on Perl 6. We chose the immediate benefits for Perl 5.
- deleted 3y ago[deleted]
- drewg123 3y ago"Congratulations. You aren't running Eunice." I still remember seeing this line from the configure script scroll by.. Made me curious what Eunice was, which was harder than it might seem, given that this was before the web and any search engines.. (and yes, it turns out Eunice was a unix-ish environment on VAX/VMS)
- tod222 3y agoYep, good times!
- kagevf 3y agoThe next time I need awk or sed, whatever I end up writing I’ll submit to chat gpt and ask it to rewrite it in perl so I can compare. I’m curious because so many people (well, enough people anyway) speak so highly of perl’s text processing capabilities. If it’s such a great tool, then I want to use it too. Edit: one plus is that perl is as ubiquitous as awk and sed. It was there the whole time and I didn’t even realize it!
- rmilk 3y agoThis sounds interesting! The main reason I moved to perl instead of sed or AWK is for better control. Shell scripts don’t differentiate between variables and data (anyone else still having nightmares from trying to escape quotes and meta chars in a shell?). Perl gives you all the simple features from sed or AWK and adds useful (maintainable) foreach iterators, arrays, hashes, etc. Recommend using Strict mode if you are new to Perl. It gives good guardrails against silly mistakes like not declaring or misspelling a variables, or accessing strings or numbers that aren’t the correct datatype.
- kagevf 3y agoThank you for the tip re: strict! Escaping - that's definitely a sore point. I feel like I get tripped up by embedded double quotes way too often - does perl have a good way to handle them?
- tod222 3y agoFrom 'man perlop': Customary Generic Meaning Interpolates '' q{} Literal no "" qq{} Literal yes -- where {} can be any bracket pair. my $string1 = q<a single quote '>; my $string2 = qq<a double quote ">;
- tod222 3y ago> anyone else still having nightmares from trying to escape quotes and meta chars in a shell? This was a major reason I downloaded Perl 1.0 off Usenet in 1987.[1] [1]https://news.ycombinator.com/item?id=36657782 https://news.ycombinator.com/item?id=36657782
- emmelaich 3y agoPerl is/was fantastic and awful. An excellent alternative to grep, awk, sed, ls, test. A lot of the greatness of Perl came from writings of Larry Wall, Mark Jason Dominus, Randal Schwartz, Tom Christiansen. There is so much wisdom to be gained from all their code and documentation.
- cutler 3y agoAbsolutely. The Perl Team stands unrivaled to this day by anything in subsequent communities. That and the O'Reilly Perl series.
- whartung 3y agoMy only real forays with Perl in anger was back with the original camel book and Perl 4. The secret sauce was the ability to persist the associative arrays to a dbm file. Even though I worked on a lot of Unix boxes doing assorted data processing, Perl was never my goto for the same reason I adopted vi instead of eMacs. As good as it was, it didn’t come stock on my clients machines, and I could not presume to install it or count on it being there. So, the litany of classic Unix tools prevailed and I simply became adept at working with those. My few temptations to dip my toes in the modern (at the time) Perl waters just found indecipherable source code (to my ignorant eyes) and disastrous attempts to get whatever it was I was dabbling with out of CPAN. I was never very successful with it.
- unixhero 3y agoI could never learn it
- asicsp 3y agoA while back, I wrote a two-part article highlighting use cases where Perl's rich regular expression engine, built-in functions, extensive ecosystem and portability helps. Includes examples that are easier to do with Perl compared to sed/awk: [0] https://www.perl.com/article/perl-one-liners-part-1/ https://www.perl.com/article/perl-one-liners-part-1/ [1] https://www.perl.com/article/perl-one-liners-part-2/ https://www.perl.com/article/perl-one-liners-part-2/
- deleted 3y ago[deleted]
- wodenokoto 3y agoReading this summarizes my feelings on what kind of new language we need. Something that makes workloads usually delegated to find, grep, sed and awk coherent and easy to use, allows streaming data between functions/commands and can still be used to glue things together. Can I use Perl instead of bash/as my command line ? Should I start learning Perl?
- thesuperbigfrog 3y ago>> Can I use Perl instead of bash/as my command line ? No. Perl is not a command line shell just as Python and Ruby are not. >> Should I start learning Perl? If you are doing workloads with the command line tools you list above (find, grep, sed and awk), yes. If you are familiar with bash, sed, and awk you will find Perl familiar-looking and very handy.
- stubish 3y agoI found Perl a practical replacement on the command line (not as the command line) to command line sed and awk, to the point I never learned sed and awk. Thanks to the -p and -e options (perl -pe 's/foo/bar/'). It never felt like a practical replacement to find. It would probably work as a grep replacement, but my fingers already knew grep. Today? Don't know. Back in the day though, in your standard heterogeneous environment it was the most practical way of working since your awk and sed seemed to be platform dependent in many ways (awk, gawk, nawk, proprietary extensions etc.)
- AbuAssar 3y agoand yet now 37 years later, awk and sed are still wildly used!
- sli 3y agoThe most lovely language that I never want to use again.
- stcroixx 3y agoI miss it. I’m too rusty to be effective with it anymore, but it’s the most pleasant to use and productive language for me for small stuff. Best community there ever was as well, mostly C and Unix system programmer types.
- cutler 3y agoMe too. Ruby comes close but doesn't have Perl Monks, CPAN Testers or the The Camel Book (Pickaxe doesn't compare).
- _8j50 3y agoThere was no Git in 1987, so was this migrated from something else to git? If so, what vcs did they use?
- louzell 3y agoThe body of the op.crypt test is pretty neat. He is clearly excited about the perf optimization: # this evaluates entirely at compile time! if (crypt('uh','oh') eq 'ohPnjpYtoi1NU') {print "ok 1\n";} else {print "not ok 1\n";} # this doesn't. $uh = 'uh'; if (crypt($uh,'oh') eq 'ohPnjpYtoi1NU') {print "ok 2\n";} else {print "not ok 2\n";}