5 ms·

Python and Perl areas of strength do not overlap, IMHO. Python is great to write well structured programs with modules, well defined classes, etc. Perl is gre

Python and Perl areas of strength do not overlap, IMHO. Python is great to write well structured programs with modules, well defined classes, etc. Perl is great to glue stuff together in place of shell scripts. I see way too often people attempting to write Python scripts to accomplish simple tasks and they inevitably end up being full of boilerplate, third party modules, stuff that breaks at every python update, etc. Perl is changing but it always stays backward compatible, it's installed literally everywhere and it has unmatched regex support, so it's super easy to run a few commands and process their output. Perl is also arguably a better awk than awk and a better sed than sed, imho. I haven't used sed in ages because 99% of the time `per -pE 's/reg/ex/g' -i <file>` is way more convenient and portable than `sed`.

sgt 3y ago
It's interesting that people say the same about Java and Python; Java is great for well structured programs and scalability. Python is great for scripts, just don't let the programs grow too much in size. (Not my opinion, I am impressed with how well Python programs can scale in size if using a proper framework e.g. Django)
- nerdponx 3y ago
  Python is excellent for "medium scale". A little too verbose for very simple ad-hoc scripts, a little too unstructured for very large applications with a large number of developers (but this has improved substantially in recent years).
  - qalmakka 3y ago
    This. Medium scale is what Python excels at - the sweet spot is IMHO between 3K-10K lines. Above that and you start running into sore points (performance, no parallelism, too much dynamic typing, ...) and you maybe start looking into writing some parts of your code in C/C++/Rust (or yet another custom runtime, like Dropbox or Google attempted to do once). Under that it's still very effective but probably not the sharpest tool around - IMHO Python excels at being a jack of all trades, but there are often more specialised languages that tend to be more effective in a specific niche (like, for instance, Perl when writing scripts that glue tools together).
- ilyt 3y ago
  I literally never heard that in 15+ years of my career. Generic sneering in direction of "scripting languages", sure.
LudwigNagasena 3y ago
With Python the scope of “simple tasks” moves from a bunch of shell scripts to eg parsing a webpage, getting images from it and running face detection on them. And if it happens so that your ad hoc script has to become a full-fledged program, you can refactor it, add API endpoints, monitoring and so on.
- qalmakka 3y ago
  > your ad hoc script has to become a full-fledged program The very second a quick and dirty script grows into a full-fledged program, it immediately becomes technical debt.
  - dkarl 3y ago
    If by "technical debt" you mean something that requires updates and maintenance, then yes, because it's a living language. Perl has the advantage of being frozen in time, but it has the disadvantages of not being a living language: younger programmers not being prepared or willing to touch it, likely less library support for newer technologies. That constitutes technical debt as well, albeit of a different kind.
  - iso1631 3y ago
    > The very second a quick and dirty script grows into a full-fledged program, it immediately becomes technical debt. It often becomes a business asset which delivers far more value to the business than any maintenance (often very low) which is needed
somat 3y ago
For me the real advantage of awk, and I love me my awk, is that the entire reference manual is 10 pages long. http://man.openbsd.org/awk http://man.openbsd.org/awk Other than that I agree, awk runs out of steam rather quickly. Shell runs out of steam fast as well(my cutoff for moving out of shell to a better language is the moment I need arrays.) But the killer feature of shell, the thing I wish was in more languages is pipes. The ability to flow one process into another process with a single character. I never did learn perl. but doing the equivalent to a pipe in python is a lot of awkward wrangling with subprocess and manually assigning streams. The way shell can pipe into and out of control flow is magical to me.
- qalmakka 3y ago
  > pipes In Perl you can just run commands using backticks or qx// and get their output, and so you can pipe them in any way you want: my @a = map { chomp; $_ } `ls -1 | sort | uniq` You can otherwise use `open` to spawn shells as if they were files, piping either on their left or right: open my $output, '-|', 'ls -1 | sort | uniq'; while (<$output>) { .. }
- dmd 3y ago
  If you want to go beyond those 10 pages, the (also very short!) "official" book by Aho, Weinberger, and Kernighan ("The AWK Programming Language") is an absolute gem of technical writing. It is a joy to read.
  - kamaal 3y ago
    While the book is awesome. And I guess there is a story some where that a programmer working with Aho even build CAD software with awk. Its often not an effort for the faint hearted, even to maintain. Without unit tests, exceptions, and myriad other features, you are doing too much of 'building universe to make apple pie' sort of work to get anywhere. Same with shell. You begin to give on it for non trivial work.
- theamk 3y ago
  I gave up on awk and always do perl one-liners. For basic stuff, they are about the same (perl _does_ have autosplit etc..) but every once in a while there is a slightly more complex task, like detect first unique field, work with json, shell quote, etc.. This is still trivial in perl, but turns out super ugly (or impossible) in awk. And if I have to fallback to perl periodically, why bother with awk at all? It probably made sense in the past, when machines were slow and awk was way more popular; but that does not matter now. Might as well go with more powerful option.
nerdponx 3y ago
> stuff that breaks at every python update Eh? Python backward compatibility is excellent. Most Python 3.3 scripts still run on 3.11. The only big breaking changes have been in asyncio, which was explicitly marked as provisional for years and if you were using it in ~3.6 you knew and were warned that things would change. Packaging is another story, but we're talking about scripting, not distributing software.
- parasense 3y ago
  > Eh? Python backward compatibility is excellent. Sure, so long as one wilfully ignores the 2.x to 3.x transition, that took 10 years. To be clear, to presume Python starts at version 3.3 is slightly weird, but we get it... One good example here is when pip stopped working for old python-2.x. > Most Python 3.3 scripts still run on 3.11. I would agree python forward compatibility is pretty good, but not so sure about backwards compatibility. When the walrus operator first appeared, there was a period of time where scripts began using the new feature, and the version of the python interpreter used in all the places that script needed to run. Same thing for type annotations. And there have been plenty of breaking module changes. > Packaging is another story, but we're talking about scripting, not distributing software. That's fair, but avoids a big aspect of the problem statement, of when Python updates.
  - rcarmo 3y ago
    People keep beating the 2.x dead horse. Moving to 3.x was a non-issue in 90% of the stuff I've seen "upgraded" written as far back as 2010 (at a telco, with a lot of legacy). I stopped writing 2.x code in 2013, and 3.5 (the version that became the new baseline there since I left, which was updated to 3.5.10 in 2020) dates from 2015. I'm now running mostly 3.10, solely because there are a few binary wheels for 3.11 that aren't yet on ARM for some reason (mostly ML stuff, which tends to be maintained by academia). Any kind of argument that the 2.x -> 3.x transition is still relevant has to take in these timescales for context. Otherwise it just looks... dated.
  - nerdponx 3y ago
    > Sure, so long as one wilfully ignores the 2.x to 3.x transition, that took 10 years. To be clear, to presume Python starts at version 3.3 is slightly weird, but we get it... One good example here is when pip stopped working for old python-2.x. The 2.x to 3.x transition was specifically an attempt at making a clean break, bookended by 10 years of compatibility before and after, so it doesn't really count here. I deliberately chose 3.3 because that's the earliest 3.x version I'm aware of that started to see serious adoption. > I would agree python forward compatibility is pretty good, but not so sure about backwards compatibility. You're right. Forward compatibility is what's good, but that's what the thread here is about. Complaining about old Python scripts breaking in new versions is a complaint about forward compatibility. Backward compatibility is not good, but older Python versions are EoL anyway and if you want to use them you're on your own. > That's fair, but avoids a big aspect of the problem statement, of when Python updates. The relevant aspect here is that older packaging tools historically have relied on components of Python that have been deprecated for years and/or internals with no stability guarantee, which are gradually now being removed, and so older packaging tools (e.g. older versions of Pip) are finally starting to break. So yes, packages with complicated setup.py scripts from 2011 probably won't work as-is. People love to complain about Python packaging tools, but the PEP 517 transition is much like the 2->3 transition in that most applications are not hard to upgrade, and most big open source packages have upgraded already. Python packaging was absolutely horrible for years, it's not surprise that something has to break in the process of making it less horrible. That most non-PEP-517 packages still install and work fine is a testament to how good forward compatibility actually is. I get really annoyed when people complain about how Python ain't what it used to be. It's better than what it used to be, and a little breakage along the way is necessary to get there.
chuckledog 3y ago
I stopped using Perl 15 years ago, except for this exact use case. I still type “perl -pi -e” to wrangle source code or data files, at least monthly. The regex syntax is more consistent too.
dunefox 3y ago
>Python is great to write well structured programs with modules, well defined classes, etc. That's news to me.

sgt 3y ago

It's interesting that people say the same about Java and Python; Java is great for well structured programs and scalability. Python is great for scripts, just don't let the programs grow too much in size. (Not my opinion, I am impressed with how well Python programs can scale in size if using a proper framework e.g. Django)

nerdponx 3y ago
Python is excellent for "medium scale". A little too verbose for very simple ad-hoc scripts, a little too unstructured for very large applications with a large number of developers (but this has improved substantially in recent years).
- qalmakka 3y ago
  This. Medium scale is what Python excels at - the sweet spot is IMHO between 3K-10K lines. Above that and you start running into sore points (performance, no parallelism, too much dynamic typing, ...) and you maybe start looking into writing some parts of your code in C/C++/Rust (or yet another custom runtime, like Dropbox or Google attempted to do once). Under that it's still very effective but probably not the sharpest tool around - IMHO Python excels at being a jack of all trades, but there are often more specialised languages that tend to be more effective in a specific niche (like, for instance, Perl when writing scripts that glue tools together).
ilyt 3y ago
I literally never heard that in 15+ years of my career. Generic sneering in direction of "scripting languages", sure.

LudwigNagasena 3y ago

With Python the scope of “simple tasks” moves from a bunch of shell scripts to eg parsing a webpage, getting images from it and running face detection on them. And if it happens so that your ad hoc script has to become a full-fledged program, you can refactor it, add API endpoints, monitoring and so on.

qalmakka 3y ago
> your ad hoc script has to become a full-fledged program The very second a quick and dirty script grows into a full-fledged program, it immediately becomes technical debt.
- dkarl 3y ago
  If by "technical debt" you mean something that requires updates and maintenance, then yes, because it's a living language. Perl has the advantage of being frozen in time, but it has the disadvantages of not being a living language: younger programmers not being prepared or willing to touch it, likely less library support for newer technologies. That constitutes technical debt as well, albeit of a different kind.
- iso1631 3y ago
  > The very second a quick and dirty script grows into a full-fledged program, it immediately becomes technical debt. It often becomes a business asset which delivers far more value to the business than any maintenance (often very low) which is needed

somat 3y ago

For me the real advantage of awk, and I love me my awk, is that the entire reference manual is 10 pages long. http://man.openbsd.org/awk http://man.openbsd.org/awk Other than that I agree, awk runs out of steam rather quickly. Shell runs out of steam fast as well(my cutoff for moving out of shell to a better language is the moment I need arrays.) But the killer feature of shell, the thing I wish was in more languages is pipes. The ability to flow one process into another process with a single character. I never did learn perl. but doing the equivalent to a pipe in python is a lot of awkward wrangling with subprocess and manually assigning streams. The way shell can pipe into and out of control flow is magical to me.

qalmakka 3y ago
> pipes In Perl you can just run commands using backticks or qx// and get their output, and so you can pipe them in any way you want: my @a = map { chomp; $_ } `ls -1 | sort | uniq` You can otherwise use `open` to spawn shells as if they were files, piping either on their left or right: open my $output, '-|', 'ls -1 | sort | uniq'; while (<$output>) { .. }
dmd 3y ago
If you want to go beyond those 10 pages, the (also very short!) "official" book by Aho, Weinberger, and Kernighan ("The AWK Programming Language") is an absolute gem of technical writing. It is a joy to read.
- kamaal 3y ago
  While the book is awesome. And I guess there is a story some where that a programmer working with Aho even build CAD software with awk. Its often not an effort for the faint hearted, even to maintain. Without unit tests, exceptions, and myriad other features, you are doing too much of 'building universe to make apple pie' sort of work to get anywhere. Same with shell. You begin to give on it for non trivial work.
theamk 3y ago
I gave up on awk and always do perl one-liners. For basic stuff, they are about the same (perl _does_ have autosplit etc..) but every once in a while there is a slightly more complex task, like detect first unique field, work with json, shell quote, etc.. This is still trivial in perl, but turns out super ugly (or impossible) in awk. And if I have to fallback to perl periodically, why bother with awk at all? It probably made sense in the past, when machines were slow and awk was way more popular; but that does not matter now. Might as well go with more powerful option.

nerdponx 3y ago

> stuff that breaks at every python update Eh? Python backward compatibility is excellent. Most Python 3.3 scripts still run on 3.11. The only big breaking changes have been in asyncio, which was explicitly marked as provisional for years and if you were using it in ~3.6 you knew and were warned that things would change. Packaging is another story, but we're talking about scripting, not distributing software.

parasense 3y ago
> Eh? Python backward compatibility is excellent. Sure, so long as one wilfully ignores the 2.x to 3.x transition, that took 10 years. To be clear, to presume Python starts at version 3.3 is slightly weird, but we get it... One good example here is when pip stopped working for old python-2.x. > Most Python 3.3 scripts still run on 3.11. I would agree python forward compatibility is pretty good, but not so sure about backwards compatibility. When the walrus operator first appeared, there was a period of time where scripts began using the new feature, and the version of the python interpreter used in all the places that script needed to run. Same thing for type annotations. And there have been plenty of breaking module changes. > Packaging is another story, but we're talking about scripting, not distributing software. That's fair, but avoids a big aspect of the problem statement, of when Python updates.
- rcarmo 3y ago
  People keep beating the 2.x dead horse. Moving to 3.x was a non-issue in 90% of the stuff I've seen "upgraded" written as far back as 2010 (at a telco, with a lot of legacy). I stopped writing 2.x code in 2013, and 3.5 (the version that became the new baseline there since I left, which was updated to 3.5.10 in 2020) dates from 2015. I'm now running mostly 3.10, solely because there are a few binary wheels for 3.11 that aren't yet on ARM for some reason (mostly ML stuff, which tends to be maintained by academia). Any kind of argument that the 2.x -> 3.x transition is still relevant has to take in these timescales for context. Otherwise it just looks... dated.
- nerdponx 3y ago
  > Sure, so long as one wilfully ignores the 2.x to 3.x transition, that took 10 years. To be clear, to presume Python starts at version 3.3 is slightly weird, but we get it... One good example here is when pip stopped working for old python-2.x. The 2.x to 3.x transition was specifically an attempt at making a clean break, bookended by 10 years of compatibility before and after, so it doesn't really count here. I deliberately chose 3.3 because that's the earliest 3.x version I'm aware of that started to see serious adoption. > I would agree python forward compatibility is pretty good, but not so sure about backwards compatibility. You're right. Forward compatibility is what's good, but that's what the thread here is about. Complaining about old Python scripts breaking in new versions is a complaint about forward compatibility. Backward compatibility is not good, but older Python versions are EoL anyway and if you want to use them you're on your own. > That's fair, but avoids a big aspect of the problem statement, of when Python updates. The relevant aspect here is that older packaging tools historically have relied on components of Python that have been deprecated for years and/or internals with no stability guarantee, which are gradually now being removed, and so older packaging tools (e.g. older versions of Pip) are finally starting to break. So yes, packages with complicated setup.py scripts from 2011 probably won't work as-is. People love to complain about Python packaging tools, but the PEP 517 transition is much like the 2->3 transition in that most applications are not hard to upgrade, and most big open source packages have upgraded already. Python packaging was absolutely horrible for years, it's not surprise that something has to break in the process of making it less horrible. That most non-PEP-517 packages still install and work fine is a testament to how good forward compatibility actually is. I get really annoyed when people complain about how Python ain't what it used to be. It's better than what it used to be, and a little breakage along the way is necessary to get there.

chuckledog 3y ago

I stopped using Perl 15 years ago, except for this exact use case. I still type “perl -pi -e” to wrangle source code or data files, at least monthly. The regex syntax is more consistent too.

dunefox 3y ago

>Python is great to write well structured programs with modules, well defined classes, etc. That's news to me.