25 ms·

Selecting a programming language can be a form of premature optimization

davesnx 5y ago
The author: Selecting a programming language can be a form of premature optimization Also the author: Pick python no matter the problem, the team, the libraries, the deployment targets, etc.
- eesmith 5y ago
  Where does the author imply something like "Pick python no matter the .. etc.?" I read it as the much more milder "don't reject Python because of vague concerns about run-time performance". I can't find anywhere which suggests the author things people should use Python to, for example, code up their web app front-ends or to implement a 'hard' real-time operating system.
  - davesnx 5y ago
    Mi bad english played me bad here. I meant to write "Picks python... blabla" Looked to me like a "Python isn't than bad for perf" article.
adibalcan 5y ago
I don’t know any business bankruptcy because of selecting wrong programming language.
retrocryptid 5y ago
Since this post has morphed into a "this is why I think language $(FOO) is good," here's another list which takes liberties with the concept of "programming language." https://meadhbh.hamrick.rocks/v2/technical/six_hot_languages_programmers_should_learn.html https://meadhbh.hamrick.rocks/v2/technical/six_hot_languages...
outsomnia 5y ago
Thesis is dogmatic selection of language is bad: article is about using Python under all circumstances.
- 4pkjai 5y ago
  Yes, I’d say replace python with [whatever language your team is most comfortable with]
- birdstheword5 5y ago
  I've met several people who learned Python and refuse to learn any other language. I wonder if this mentality exists for other languages
  - exyi 5y ago
    Oh yes, it's sooo common. I met many C# web developers who just refuse to even learn a bit of JS, "because it's soo bad language"
    - Semaphor 5y ago
      I think JS and PHP have a special status here and don’t count as an answer to what GP was asking. Many people hate JS and/or PHP, often for legacy reasons.
      - exyi 5y ago
        But it's not only JS, I used it as an example because these people are doing websites, yet don't even want to learn JS. They also hate to write SQL and are stuck with ORMs...
        Semaphor 5y ago
        That is different, but SQL is also not a general programming language. I see what you say as an issue with those people, but a different issue ;)
resonious 5y ago
It'd be great if we could start measuring how much $ it costs to run our services vs how much developer salary $ it costs to maintain and build them. If Rust or something really saves money because it means we don't need 50 extra web workers for the same TPS then I don't think it's "premature optimization" - unless the dev salary is too much! As it stands we're all just saying stuff with no data or evidence to back it up.
- Karrot_Kream 5y ago
  It's not going to change any time soon, IMHO. Programmers have too big of a culture of individuality, blogging, and intellectual daydreaming to actually try to come to consensus on terms of art. On top of which businesses are loathe to share data because it's tied very closely to their competitive advantage. Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Until this changes, it'll continue to be discordant voices talking over each other. Ideology [1] is a great talk about this. [1]: https://www.destroyallsoftware.com/talks/ideology https://www.destroyallsoftware.com/talks/ideology
  - Jensson 5y ago
    > Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Sounds exactly like business management culture. Makes sense though, developer productivity is first and foremost a business management issue so discussions about it ought to follow the same path as business management. And business management love their anecdotes, great person citations and jargon.
- wongarsu 5y ago
  With silicon valley salaries maybe the trade-off is more between Developer time vs. Devops time. Those 50 extra web workers might not matter much on their own, but someone has to set them up, orchestrate them and deal with all weird interactions that arise. And of course the further you get from silicon valley the more the hardware/service costs matter. A developer in Sofia easily earns an order of magnitude less than one in San Francisco, which shifts the whole optimization tradeoff a lot.
lewisjoe 5y ago
Very true. People discriminate against dynamic languages with usually two measurements: runtime errors and performance. But we often overlook the fact that it's possible to scale those two, incrementally. With today's convenience in interop, we can always start with a prototype friendly technology and later switch tech for real bottlenecks.
revskill 5y ago
Agree with title. In 99% of business use cases, typescript/nodejs/reactjs is enough.
number6 5y ago
Writing Code at all can be a Form of premature optimization
- mro_name 5y ago
  …or even doing it with computers. Or even doing it at all.
  - abrichr 5y ago
    When it comes to starting a business, this is very good and often overlooked advice.
- eesmith 5y ago
  Yeah, I remember working on one of the Project Euler problems, getting the right answer, then seeing someone else had solved it with a pocket calculator in less time than I took to write the program.
  - JohnWhigham 5y ago
    Ain't it funny how these programming exercise websites are essentially the adult versions of busywork?
    - mro_name 5y ago
      or play. Maybe that's it. Work without purpose. I like the busywork. Less spooky than 'cultic ritual'.
  - deckard1 5y ago
    It was a rather embarrassing realization for me that the Fibonacci sequence has a closed-form expression and can be calculated without an algorithm. I had always seen it done in pedagogical fashion as an exercise in explaining recursion and memoization and just assumed it had to be done that way. The downside of using toy/straw problems...
brabel 5y ago
This blog post assumes Python is more productive than other languages. People like to make this claim about not only Python, but many other dynamic programming languages, specially Ruby, PHP and Lisp... but there's very little evidence to support that... programmers tend to be most productive in whatever language they know best. If they know both Python and Java equally well, I would bet they would be almost exactly as productive in either.
- rambojazz 5y ago
  Granted, people come from different backgrounds, and "the right tool for the right job" always applies. But it's undeniable that generally speaking, ie. not considering any specific cases or requirements, some languages are definitely more productive than others.
- josephg 5y ago
  I doubt it, though if anyone has data I’d love to see it. At the moment I’m quite comfortable in both javascript and rust. But I find javascript (/ typescript) noticeably more productive for small to medium projects. I can get more done with less work. For a variety of reasons it seems to take more work to write good rust code than good javascript code. This isn’t a knock on rust - I just think rust trades off programmer productivity for correctness and performance. And it shows, on both sides.
  - KronisLV 5y ago
    > I doubt it, though if anyone has data I’d love to see it. I recall a study being quoted in my university courses, which described the amount of code needed to get certain things done between different languages, which showed that Python, Ruby and others are on the less verbose side, the reasoning being that on average they'd also be more productive. This seems to coincide with my personal experience, where JavaScript with React was much easier to work with in smaller projects, whereas using TypeScript with React lead to much slower development because of all the typing that needed to be handled, especially in cases of union types. Now, one can say that it's worth the effort to be more confident in your code doing what you want it to both now and after X months, much like you could sometimes prefer the type systems of Java or .NET over Python, Ruby or PHP, but in my eyes those tradeoffs always come with slower development velocity. The sad thing, however, is that DuckDuckGo (and possibly other search engines) failed to return the original study or anything like it, only resulting in low quality blog content: - https://duckduckgo.com/?t=ffab&q=programming+language+comparison+amount+of+code - https://duckduckgo.com/?q=programming+language+comparison+verbosity - https://duckduckgo.com/?q=programming+language+lines+of+code+java+ruby - http://libgen.is/scimag/?q=programming+language+verbosity - http://libgen.is/scimag/?q=programming+language+lines+of+code Anyone have any better search queries for this? Any idea why the search result quality is generally so low? Any ideas which study it might have been referencing?
throwaway_2047 5y ago
Learning coding can be a form of premature optimization. Exercising can be a form of premature optimization. Your body ain't gonna need it. jk
mobilemidget 5y ago
I'm a big fan of the "Make It Work Make It Right Make It Fast" If one of these steps requires a change of programming language, I'm never too lazy to recode :)
- Jensson 5y ago
  Problem is that if you follow that strategy then it will likely never be as performant as if you did "Make It Fast, Make it work". So if you know that performance is a key factor to success then you should follow this path instead. And by "make it fast, make it work", I mean benchmark first. Kind of like test first, you write the benchmark before the implementation and constantly look at how fast things runs, and ensues every single bit you add is fast.
  - DemocracyFTW 5y ago
    I don't think the "Make it Right" part should or can reasonably be left out of the equation, otherwise I could just compute something if all you care about is speed. That said, I have used a performance-first approach several times using own old and new solutions and libraries and have found early benchmarks to be a great tool to weed out untenable (i.e. order-of-magnitude slower) stuff. This then can mean I have to write fewer tests to make plausible my own or a 3rd party solution does in fact what I expect it to do. At a certain point performance becomes correctness.
firasd 5y ago
Just been thinking through this... I'm working on a personal project (podcast indexing/search) that involves parsing a lot of RSS feeds. Some years ago I had built the feed checker in Elixir but now when I tried to get it going again I was having too much trouble with version and compatibility changes in Phoenix. I eventually just did that part in PHP cause that's my day-to-day language. Wouldn't have made sense to block all development until I re-learned how to query a database in Phoenix. Plus later I can re-implement that feed checking part as as service in Elixir or another language. First make it work..
srvmshr 5y ago
I quite believe that every programming language has its strengths & weaknesses, and a few languages is a must in every developer's accoutrements. Want to build an I/O utility writing to a DB? Sure C can do it, but Python is better suited. Want to write a toy compiler? You don't want to waste your time trying to wrangle on CPython extensions. C works out of the box. > _'if you select a programming language based on your preconceived notions of how a language performs, you will never know if the language that might be a better, more productive fit'_ Part of the CS education is not about recognizing homeruns but understanding trade-offs. The experience gain is about learning how tools work & which tools to choose to work in tandem. Modern systems use a variety of languages - JS in the webpage, SQL DBMS for queries, C++ to run the performance bits, Python for ML, introperations - maybe even Rust in the security bits of late. In that sense, the title was unfortunately misleading to me, since author tried to demonstrate a lot of usecases with Python. Python is great - but there has to be a reason why other languages co-exist. Not just for bankers, military or some enthusiastic hobbyist.
scottcodie 5y ago
People are too shy on hardware costs. Many of my professional colleagues develop and optimize software full time that runs on a server that is as fast as the phone in their pocket.
exyi 5y ago
I don't like how he makes it a choice between Python and C++/Rust. There is very many languages that are more similar to Python in convenience and yet run reasonably fast (and you can actually optimize some procedures when you need to, because there is a compiler). Go, Julia, C#, F#, Scala, Kotlin, even recent Java... all much faster than Python and much less pain to work with than C++. And the interoperability is not as awesome as it's painted in the article, it's always more pain to have more languages in a project that need to talk together
- raxxorrax 5y ago
  You have to look at resources too. You probably find 10 Java developers before you find 1 Scala developer, even if they are related technically. Especially on long term projects you have at least some turnover of people.
qaq 5y ago
I am less productive in Python vs say Go. So by default use Python would be a bad heuristic for people like me. Has nothing to do with runtime performance for the most part.
pella 5y ago
> Prototype in Python Julia one of the target is solving the "two-language problem" "Julia seemed to have solved the “two-language problem”—a conundrum often facing Python programmers, as well as users of other expressive, interpreted languages. You write a program to solve a problem in Python, enjoying its pleasant syntax and interactivity. The program works on a test version of your problem, but when you try to scale it up to something more realistic, it’s too slow. This is not your fault. Python is inherently slow—something that doesn’t matter for some types of applications but does matter for your big simulation. After applying various techniques to speed it up but only realizing modest gains, you finally resort to rewriting the most time-consuming parts of the calculation in C (most commonly). Now it’s fast enough, but now you also need to maintain code in both languages, hence the two-language problem." https://arstechnica.com/science/2020/10/the-unreasonable-effectiveness-of-the-julia-programming-language/ https://arstechnica.com/science/2020/10/the-unreasonable-eff...
- GavinMcG 5y ago
  Nim seems like another candidate.
- deckard1 5y ago
  What's the tl;dr of how Julia is solving this? Looking around it seems the answer is "multiple dispatch". Which seems suspect considering many languages have already tried this (Common Lisp, for example). > Clearly, multiple dispatch, or some other way around the expression problem, is necessary for the kind of fluent composability that I’ve described above—but it is not sufficient. Julia has enjoyed an explosive degree of uptake in the scientific community because it combines this feature with several others that make it very attractive to numericists. That's incredibly handwavy. So what's the special sauce? There is no such thing as a free lunch when it comes to dynamic vs. static. It also seems like Julia is trading off expressiveness and easy of use in favor of efficiency, based on comments from people that have used Julia. It's one thing to be faster than any inherently slow language (Ruby, Python, Smalltalk, etc.), but keeping that flexibility and being as fast as C/C++ is a rather bold claim. Most languages hit some middle ground between the two, such as Java. But no one is under the delusion that trade-offs weren't made to get there.
deleted 5y ago
[deleted]
IshKebab 5y ago
"Guys, our Python prototype is finished. It works really well and has proved our concept. I know we've put a ton of work into it but now I suggest we scrap it and rewrite it in a language that isn't dog slow." - Nobody
- geofft 5y ago
  Largely because nobody has ever found Python to be dog slow. Put your unoptimized Python into production, you'll be fine.
  - gpderetta 5y ago
    We run pylint as part of our build, including as a prerequisite of merging a branch into master. Our codebase is well over 1M lines of C++. We have about 100k lines of python. Running pylint takes the same order of magnitude of time (half as much IIRC) as running a full optimizing build + linking of the C++ code base. We run pylint over all cores, while we run the C++ build only on a subset of cores on the build machine. I would call that dog slow, unless you think python is not the appropriate language write pylint. And no, it is not fine.
  - IshKebab 5y ago
    Dropbox is the biggest user of Python that I know of, and they went as far as writing their own JIT compiler before giving up and switching to faster languages.
KingOfCoders 5y ago
This is always the case with me, once with Scala, now with Rust.
agilob 5y ago
>Selecting a programming language can be a form of premature optimization >Prototype in Python. Stopped reading here. Wait, what? The only thing I know about python is that it's indentation-sensitive, no idea about syntax or libraries. Suggesting me to use python is premature optimisation.
- geofft 5y ago
  If you continued reading past the point where you didn't know things, you would know more things. :)
j4mie 5y ago
My (perhap controversial) take is that static typing in Python is also a form of premature optimisation. Code should be written without static types first, and static types should only be added when absolutely necessary. 99% of the time, it never will be.
- IshKebab 5y ago
  Why though? Static types make it easier to write code overall, so you're slowing yourself down for no real benefit. You could just as easily say "Descriptive variable names are a form of premature optimisation. Code should be written with single-letter names, and descriptive names only added when absolutely necessary."
raxxorrax 5y ago
Selecting a language is like selecting a tool. It would be worse to select a language that on first sight fits the problem if you don't have any experience with it. To be honest, today there are many viable solutions with different languages. I would not recommend to start with a plan where you have to reimplement the system in another language at some point. Experience tells me that almost none of such projects survive.
olivierduval 5y ago
Actually, writing a POC or a MVP is quite different to writing a "long run" production product. In the first case, coding speed is surely a must - and any untyped (or loosely typed) language might be good enough - but in the last case, when dev teams have to maintain some code in the long run, during multiple versions, with turnover, typed (or pedantic) language might be easier to work with because the language already include some kind of documentation (types) and automatic checks (type verification). Moreover, the availability of "average" (and "cheap") programmers matters a lot in the long run: if only genius can maintain your system, then you'll have problem in the long run because you'll either need to keep them at all price or need a lot of time to replace them. So, in the long run, you should better use a wide audience language with a lot of available programmer (even if they are "average") than a specific language requiring good programmers. However, for an MVP, you can recruit a genius programmer using the fastest tool for the job. Obviously, some domain are more oriented toward some language... and for ML for example, python is quite a good choice because of the libs (as Java could be for - lets say web servers) So it matters a lot what your system will be used for and how much time you will require it to run before needing to rewrite it from scratch
streamofdigits 5y ago
Over time I've come to conclude we are not optimizing for language features, we are optimizing for the community around a programming language or stack. Sure, features are important and if critical ones are missing it might be a show stopper. So there is an initial thresshold that all candidates must pass. But problem solving is not a one-off exercise. It tends to be both dynamic (=facing unpredictable challenges) and recurring over long time horizons. Which means having a healthy, engaged, resourced community that will invest in adapting / solving future requirements is essential. So the "optimization" problem includes quite a bit more than the presently known developer team, its software stack its hardware and current problem definition / user requirement. I think you see this dynamic in several cases (including python) where you might not think that it makes rational sense.
StefanKiss 5y ago
well if you combine the title with the contents you reach the conclusion that python is not a programing language. fair enough.
pphysch 5y ago
It's the usual trade-off of individual programmer freedom/expressiveness vs. rigid & predictable standards that enable productivity at scale. Just because you can program a new feature quickly doesn't mean it is engineered well for the long-term & larger scales.
woah 5y ago
I thought this was going to be satire, but no, it seems that he is completely serious.
- saila 5y ago
  Is the premise unreasonable? Of course, it depends on what you're doing--there are certainly use cases where you know Python or a similar language wouldn't be appropriate (for performance or other reasons)--but it seems to me that there are plenty of scenarios where using a "faster" language up front isn't warranted, and you may never need to switch.
hsn915 5y ago
The fallacy of premature optimization rears its ugly head again. When Knuth wrote that quote, he meant something very specific with the word optimization: low level micro optimizations. Spending a lot of time trying to get every little ounce of performance from every little CPU instruction. High level reasonable design decisions are not "optimizations" in this sense at all. You should think of them as non-pessimization. Choosing Python or another slow language is premature pessimization. You just make your program slower for no reason. So choosing a language based on how programs written in it perform is simply non-pessimization. If you are interested in an expansion on this idea, checkout Casey Muratori's lecture about philosophies of optimization: https://www.youtube.com/watch?v=pgoetgxecw8 https://www.youtube.com/watch?v=pgoetgxecw8 The article makes claims about software being "fast enough" and not needing further optimization. But what you call "fast enough" is probably 10000 times slower than what it could be. I'm not joking. People who program in slow languages have a really skewed perception about performance. If a page takes 5 seconds to load, they don't see that as a problem. If a server takes 500ms to respond to a request, they don't see that as a problem. This is simply unacceptable.
- dragonwriter 5y ago
  > High level reasonable design decisions Viewing language choice as a high level design decision for a project of nontrivial scale is a symptom of bad high-level design decisions.
- flohofwoe 5y ago
  OTH not choosing Python for tasks where the language performance really doesn't matter at all (for instance shell scripting stuff) would also be futile. It's simply a matter of using the right tool for a specific job (and in Python's case, that 'tool' isn't even the language, but the batteries-included stdlib).
  - majou 5y ago
    OK but shell is cuter than python.
  - hsn915 5y ago
mumblemumble 5y ago
A case study from my own personal experience: We generally do everything in Java. Python is avoided because of all the usual complaints - dynamically typed, not fast enough, GIL, etc. So, as a PoC, I decided to try rewriting one of our services in Python. And, compared to the Java one, it is: Cheaper to write and maintain. About 1/10 as many SLOC. About 1/20 as many person-hours. Has better static type checks. For example, Java's type checker cannot statically verify for null safety. The Python one I chose does do that. Note that, since I did choose to put type hints on everything, the productivity boost in question cannot be attributed to dynamic typing. (That said, not all 3rd-party libraries have type hints, so, if you want to type check everything, you may have some yak shaving to do.) Uses less RAM. About 1/2 as much. Is faster. Admittedly I'm leaning on numpy, Cython, and friends for this. It may well be much slower for projects where that is not possible. But still, I think that the point about premature operation stands in this case. (Disclaimer: I also have a colleague at $FAMOUS_COMPANY who tells me they are moving off of Python because they found the opposite of the above in many cases. Though I personally suspect that a mitigating factor is that their profit margins and scale are large enough that all the coefficients in their cost/benefit formula are wildly different from the norm.)
- killingtime74 5y ago
  Kotlin Scala does all these and interface with Java to boot
- TheDudeMan 5y ago
  > Java's type checker cannot statically verify for null safety. Check out NullAway. > Uses less RAM. About 1/2 as much. Were you on a modern JVM?
  - mumblemumble 5y ago
    It's not just null. For example, Java much more frequently leaves you in a situation where you need to cast to/from `Object`. In Python, `Any` type hints can usually be avoided, because of union types.
GuB-42 5y ago
Python is not just slow, it also tends to break down for large projects, like all dynamic languages. It is not an absolute, you can do it, but the larger the code base, the more you need strong guarantees over flexibility, and the less relevant languages like Python tend to become. Prototyping in Python is a viable strategy. Personally I tend to use Perl for that, because I am comfortable with it and I find it better adapted to quick prototyping than Python. I then rewrite in another language, usually C++, with more care about data structures and optimization. The author strategy to do everything in Python than optimize the slow parts can make sense, though I tend to prefer the opposite: white your framework/engine in a static language (like C++) and embed an interpreter (Python, LUA, etc...), as it is common in game dev. But all that are language decisions! Whatever you do, there are going to be consequences. Choosing Python with C/C++/Rust optimizations is not bad, but it is a rather strong opinion, not the obvious default choice the author makes it.
- hsn915 5y ago
  > it also tends to break down for large projects You also end up spending a lot of time and resources in the later stages of the project trying to work around the problems: hire more people to work on the code base, spend a lot of time investigating optimization strategies, hire a devops team to turn the program into a distributed system with maybe hundreds of instances running on CloudProvider (TM)
  - api 5y ago
    > turn the program into a distributed system with maybe hundreds of instances running on CloudProvider (TM) ... and give tens to hundreds of thousands a month to cloud providers when you could spend 1/100th if it were written in Go.
    - hsn915 5y ago
      You could even save the costs by buying your own server. I doubt that you can't buy a very decent server hardware for no more than $10000. Some places pay more than that per month to Amazon.
dragonwriter 5y ago
I agree with the broad concept (with the caveat that there are exceptions to all generalizations), but note that “Python” can generally be replaced with “whatever general purpose language you are comfortable and familiar with that doesn't have any clear insurmountable barriers like ‘is not available for the desired target platform’.”
everyone 5y ago
Just use what you know. Something you've used before.
letwhile 5y ago
While I see a point in this article, I personally run into performance problems with python at a very early stage nearly every time. The overhead of starting to use some ffi is huge. Suddenly you have two projects, with two different build systems etc. Way faster to just use a performant language. Besides this, dynamic typing gets frustrating quickly, as the loc rise, and distribution is not very fun. I love the language, but won't use it anymore for any "serious" project.
eptcyka 5y ago
I really don't find writing in Python to be that much more productive than using many other modern language.
xg15 5y ago
This sounds like the "Java can be fast" arguments from two decades or the "JavaScript can be fast" arguments from one decade ago, now with the next language. I mean, sure I can prototype in python, then put a lot of additional effort into bringing performance of the bits I absolutely need fast close to native code performance. Or I could code everything in C or Rust and get everything in native code from the get-go.
deleted 5y ago
[deleted]

The author: Selecting a programming language can be a form of premature optimization Also the author: Pick python no matter the problem, the team, the libraries, the deployment targets, etc.

eesmith 5y ago
Where does the author imply something like "Pick python no matter the .. etc.?" I read it as the much more milder "don't reject Python because of vague concerns about run-time performance". I can't find anywhere which suggests the author things people should use Python to, for example, code up their web app front-ends or to implement a 'hard' real-time operating system.
- davesnx 5y ago
  Mi bad english played me bad here. I meant to write "Picks python... blabla" Looked to me like a "Python isn't than bad for perf" article.

adibalcan 5y ago

I don’t know any business bankruptcy because of selecting wrong programming language.

retrocryptid 5y ago

Since this post has morphed into a "this is why I think language $(FOO) is good," here's another list which takes liberties with the concept of "programming language." https://meadhbh.hamrick.rocks/v2/technical/six_hot_languages_programmers_should_learn.html https://meadhbh.hamrick.rocks/v2/technical/six_hot_languages...

outsomnia 5y ago

Thesis is dogmatic selection of language is bad: article is about using Python under all circumstances.

4pkjai 5y ago
Yes, I’d say replace python with [whatever language your team is most comfortable with]
birdstheword5 5y ago
I've met several people who learned Python and refuse to learn any other language. I wonder if this mentality exists for other languages
- exyi 5y ago
  Oh yes, it's sooo common. I met many C# web developers who just refuse to even learn a bit of JS, "because it's soo bad language"
  - Semaphor 5y ago
    I think JS and PHP have a special status here and don’t count as an answer to what GP was asking. Many people hate JS and/or PHP, often for legacy reasons.
    - exyi 5y ago
      But it's not only JS, I used it as an example because these people are doing websites, yet don't even want to learn JS. They also hate to write SQL and are stuck with ORMs...
      - Semaphor 5y ago
        That is different, but SQL is also not a general programming language. I see what you say as an issue with those people, but a different issue ;)

resonious 5y ago

It'd be great if we could start measuring how much $ it costs to run our services vs how much developer salary $ it costs to maintain and build them. If Rust or something really saves money because it means we don't need 50 extra web workers for the same TPS then I don't think it's "premature optimization" - unless the dev salary is too much! As it stands we're all just saying stuff with no data or evidence to back it up.

Karrot_Kream 5y ago
It's not going to change any time soon, IMHO. Programmers have too big of a culture of individuality, blogging, and intellectual daydreaming to actually try to come to consensus on terms of art. On top of which businesses are loathe to share data because it's tied very closely to their competitive advantage. Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Until this changes, it'll continue to be discordant voices talking over each other. Ideology [1] is a great talk about this. [1]: https://www.destroyallsoftware.com/talks/ideology https://www.destroyallsoftware.com/talks/ideology
- Jensson 5y ago
  > Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Sounds exactly like business management culture. Makes sense though, developer productivity is first and foremost a business management issue so discussions about it ought to follow the same path as business management. And business management love their anecdotes, great person citations and jargon.
wongarsu 5y ago
With silicon valley salaries maybe the trade-off is more between Developer time vs. Devops time. Those 50 extra web workers might not matter much on their own, but someone has to set them up, orchestrate them and deal with all weird interactions that arise. And of course the further you get from silicon valley the more the hardware/service costs matter. A developer in Sofia easily earns an order of magnitude less than one in San Francisco, which shifts the whole optimization tradeoff a lot.

lewisjoe 5y ago

Very true. People discriminate against dynamic languages with usually two measurements: runtime errors and performance. But we often overlook the fact that it's possible to scale those two, incrementally. With today's convenience in interop, we can always start with a prototype friendly technology and later switch tech for real bottlenecks.

revskill 5y ago

Agree with title. In 99% of business use cases, typescript/nodejs/reactjs is enough.

number6 5y ago

Writing Code at all can be a Form of premature optimization

mro_name 5y ago
…or even doing it with computers. Or even doing it at all.
- abrichr 5y ago
  When it comes to starting a business, this is very good and often overlooked advice.
eesmith 5y ago
Yeah, I remember working on one of the Project Euler problems, getting the right answer, then seeing someone else had solved it with a pocket calculator in less time than I took to write the program.
- JohnWhigham 5y ago
  Ain't it funny how these programming exercise websites are essentially the adult versions of busywork?
  - mro_name 5y ago
    or play. Maybe that's it. Work without purpose. I like the busywork. Less spooky than 'cultic ritual'.
- deckard1 5y ago
  It was a rather embarrassing realization for me that the Fibonacci sequence has a closed-form expression and can be calculated without an algorithm. I had always seen it done in pedagogical fashion as an exercise in explaining recursion and memoization and just assumed it had to be done that way. The downside of using toy/straw problems...

brabel 5y ago

This blog post assumes Python is more productive than other languages. People like to make this claim about not only Python, but many other dynamic programming languages, specially Ruby, PHP and Lisp... but there's very little evidence to support that... programmers tend to be most productive in whatever language they know best. If they know both Python and Java equally well, I would bet they would be almost exactly as productive in either.

rambojazz 5y ago
Granted, people come from different backgrounds, and "the right tool for the right job" always applies. But it's undeniable that generally speaking, ie. not considering any specific cases or requirements, some languages are definitely more productive than others.
josephg 5y ago
I doubt it, though if anyone has data I’d love to see it. At the moment I’m quite comfortable in both javascript and rust. But I find javascript (/ typescript) noticeably more productive for small to medium projects. I can get more done with less work. For a variety of reasons it seems to take more work to write good rust code than good javascript code. This isn’t a knock on rust - I just think rust trades off programmer productivity for correctness and performance. And it shows, on both sides.
- KronisLV 5y ago
  > I doubt it, though if anyone has data I’d love to see it. I recall a study being quoted in my university courses, which described the amount of code needed to get certain things done between different languages, which showed that Python, Ruby and others are on the less verbose side, the reasoning being that on average they'd also be more productive. This seems to coincide with my personal experience, where JavaScript with React was much easier to work with in smaller projects, whereas using TypeScript with React lead to much slower development because of all the typing that needed to be handled, especially in cases of union types. Now, one can say that it's worth the effort to be more confident in your code doing what you want it to both now and after X months, much like you could sometimes prefer the type systems of Java or .NET over Python, Ruby or PHP, but in my eyes those tradeoffs always come with slower development velocity. The sad thing, however, is that DuckDuckGo (and possibly other search engines) failed to return the original study or anything like it, only resulting in low quality blog content: - https://duckduckgo.com/?t=ffab&q=programming+language+comparison+amount+of+code - https://duckduckgo.com/?q=programming+language+comparison+verbosity - https://duckduckgo.com/?q=programming+language+lines+of+code+java+ruby - http://libgen.is/scimag/?q=programming+language+verbosity - http://libgen.is/scimag/?q=programming+language+lines+of+code Anyone have any better search queries for this? Any idea why the search result quality is generally so low? Any ideas which study it might have been referencing?

throwaway_2047 5y ago

Learning coding can be a form of premature optimization. Exercising can be a form of premature optimization. Your body ain't gonna need it. jk

mobilemidget 5y ago

I'm a big fan of the "Make It Work Make It Right Make It Fast" If one of these steps requires a change of programming language, I'm never too lazy to recode :)

Jensson 5y ago
Problem is that if you follow that strategy then it will likely never be as performant as if you did "Make It Fast, Make it work". So if you know that performance is a key factor to success then you should follow this path instead. And by "make it fast, make it work", I mean benchmark first. Kind of like test first, you write the benchmark before the implementation and constantly look at how fast things runs, and ensues every single bit you add is fast.
- DemocracyFTW 5y ago
  I don't think the "Make it Right" part should or can reasonably be left out of the equation, otherwise I could just compute something if all you care about is speed. That said, I have used a performance-first approach several times using own old and new solutions and libraries and have found early benchmarks to be a great tool to weed out untenable (i.e. order-of-magnitude slower) stuff. This then can mean I have to write fewer tests to make plausible my own or a 3rd party solution does in fact what I expect it to do. At a certain point performance becomes correctness.

firasd 5y ago

Just been thinking through this... I'm working on a personal project (podcast indexing/search) that involves parsing a lot of RSS feeds. Some years ago I had built the feed checker in Elixir but now when I tried to get it going again I was having too much trouble with version and compatibility changes in Phoenix. I eventually just did that part in PHP cause that's my day-to-day language. Wouldn't have made sense to block all development until I re-learned how to query a database in Phoenix. Plus later I can re-implement that feed checking part as as service in Elixir or another language. First make it work..

srvmshr 5y ago

I quite believe that every programming language has its strengths & weaknesses, and a few languages is a must in every developer's accoutrements. Want to build an I/O utility writing to a DB? Sure C can do it, but Python is better suited. Want to write a toy compiler? You don't want to waste your time trying to wrangle on CPython extensions. C works out of the box. > _'if you select a programming language based on your preconceived notions of how a language performs, you will never know if the language that might be a better, more productive fit'_ Part of the CS education is not about recognizing homeruns but understanding trade-offs. The experience gain is about learning how tools work & which tools to choose to work in tandem. Modern systems use a variety of languages - JS in the webpage, SQL DBMS for queries, C++ to run the performance bits, Python for ML, introperations - maybe even Rust in the security bits of late. In that sense, the title was unfortunately misleading to me, since author tried to demonstrate a lot of usecases with Python. Python is great - but there has to be a reason why other languages co-exist. Not just for bankers, military or some enthusiastic hobbyist.

scottcodie 5y ago

People are too shy on hardware costs. Many of my professional colleagues develop and optimize software full time that runs on a server that is as fast as the phone in their pocket.

exyi 5y ago

I don't like how he makes it a choice between Python and C++/Rust. There is very many languages that are more similar to Python in convenience and yet run reasonably fast (and you can actually optimize some procedures when you need to, because there is a compiler). Go, Julia, C#, F#, Scala, Kotlin, even recent Java... all much faster than Python and much less pain to work with than C++. And the interoperability is not as awesome as it's painted in the article, it's always more pain to have more languages in a project that need to talk together

raxxorrax 5y ago
You have to look at resources too. You probably find 10 Java developers before you find 1 Scala developer, even if they are related technically. Especially on long term projects you have at least some turnover of people.

qaq 5y ago

I am less productive in Python vs say Go. So by default use Python would be a bad heuristic for people like me. Has nothing to do with runtime performance for the most part.

pella 5y ago

> Prototype in Python Julia one of the target is solving the "two-language problem" "Julia seemed to have solved the “two-language problem”—a conundrum often facing Python programmers, as well as users of other expressive, interpreted languages. You write a program to solve a problem in Python, enjoying its pleasant syntax and interactivity. The program works on a test version of your problem, but when you try to scale it up to something more realistic, it’s too slow. This is not your fault. Python is inherently slow—something that doesn’t matter for some types of applications but does matter for your big simulation. After applying various techniques to speed it up but only realizing modest gains, you finally resort to rewriting the most time-consuming parts of the calculation in C (most commonly). Now it’s fast enough, but now you also need to maintain code in both languages, hence the two-language problem." https://arstechnica.com/science/2020/10/the-unreasonable-effectiveness-of-the-julia-programming-language/ https://arstechnica.com/science/2020/10/the-unreasonable-eff...

GavinMcG 5y ago
Nim seems like another candidate.
deckard1 5y ago
What's the tl;dr of how Julia is solving this? Looking around it seems the answer is "multiple dispatch". Which seems suspect considering many languages have already tried this (Common Lisp, for example). > Clearly, multiple dispatch, or some other way around the expression problem, is necessary for the kind of fluent composability that I’ve described above—but it is not sufficient. Julia has enjoyed an explosive degree of uptake in the scientific community because it combines this feature with several others that make it very attractive to numericists. That's incredibly handwavy. So what's the special sauce? There is no such thing as a free lunch when it comes to dynamic vs. static. It also seems like Julia is trading off expressiveness and easy of use in favor of efficiency, based on comments from people that have used Julia. It's one thing to be faster than any inherently slow language (Ruby, Python, Smalltalk, etc.), but keeping that flexibility and being as fast as C/C++ is a rather bold claim. Most languages hit some middle ground between the two, such as Java. But no one is under the delusion that trade-offs weren't made to get there.

deleted 5y ago

[deleted]

IshKebab 5y ago

"Guys, our Python prototype is finished. It works really well and has proved our concept. I know we've put a ton of work into it but now I suggest we scrap it and rewrite it in a language that isn't dog slow." - Nobody

geofft 5y ago
Largely because nobody has ever found Python to be dog slow. Put your unoptimized Python into production, you'll be fine.
- gpderetta 5y ago
  We run pylint as part of our build, including as a prerequisite of merging a branch into master. Our codebase is well over 1M lines of C++. We have about 100k lines of python. Running pylint takes the same order of magnitude of time (half as much IIRC) as running a full optimizing build + linking of the C++ code base. We run pylint over all cores, while we run the C++ build only on a subset of cores on the build machine. I would call that dog slow, unless you think python is not the appropriate language write pylint. And no, it is not fine.
- IshKebab 5y ago
  Dropbox is the biggest user of Python that I know of, and they went as far as writing their own JIT compiler before giving up and switching to faster languages.

KingOfCoders 5y ago

This is always the case with me, once with Scala, now with Rust.

agilob 5y ago

>Selecting a programming language can be a form of premature optimization >Prototype in Python. Stopped reading here. Wait, what? The only thing I know about python is that it's indentation-sensitive, no idea about syntax or libraries. Suggesting me to use python is premature optimisation.

geofft 5y ago
If you continued reading past the point where you didn't know things, you would know more things. :)

j4mie 5y ago

My (perhap controversial) take is that static typing in Python is also a form of premature optimisation. Code should be written without static types first, and static types should only be added when absolutely necessary. 99% of the time, it never will be.

IshKebab 5y ago
Why though? Static types make it easier to write code overall, so you're slowing yourself down for no real benefit. You could just as easily say "Descriptive variable names are a form of premature optimisation. Code should be written with single-letter names, and descriptive names only added when absolutely necessary."

raxxorrax 5y ago

Selecting a language is like selecting a tool. It would be worse to select a language that on first sight fits the problem if you don't have any experience with it. To be honest, today there are many viable solutions with different languages. I would not recommend to start with a plan where you have to reimplement the system in another language at some point. Experience tells me that almost none of such projects survive.

olivierduval 5y ago

Actually, writing a POC or a MVP is quite different to writing a "long run" production product. In the first case, coding speed is surely a must - and any untyped (or loosely typed) language might be good enough - but in the last case, when dev teams have to maintain some code in the long run, during multiple versions, with turnover, typed (or pedantic) language might be easier to work with because the language already include some kind of documentation (types) and automatic checks (type verification). Moreover, the availability of "average" (and "cheap") programmers matters a lot in the long run: if only genius can maintain your system, then you'll have problem in the long run because you'll either need to keep them at all price or need a lot of time to replace them. So, in the long run, you should better use a wide audience language with a lot of available programmer (even if they are "average") than a specific language requiring good programmers. However, for an MVP, you can recruit a genius programmer using the fastest tool for the job. Obviously, some domain are more oriented toward some language... and for ML for example, python is quite a good choice because of the libs (as Java could be for - lets say web servers) So it matters a lot what your system will be used for and how much time you will require it to run before needing to rewrite it from scratch

streamofdigits 5y ago

Over time I've come to conclude we are not optimizing for language features, we are optimizing for the community around a programming language or stack. Sure, features are important and if critical ones are missing it might be a show stopper. So there is an initial thresshold that all candidates must pass. But problem solving is not a one-off exercise. It tends to be both dynamic (=facing unpredictable challenges) and recurring over long time horizons. Which means having a healthy, engaged, resourced community that will invest in adapting / solving future requirements is essential. So the "optimization" problem includes quite a bit more than the presently known developer team, its software stack its hardware and current problem definition / user requirement. I think you see this dynamic in several cases (including python) where you might not think that it makes rational sense.

StefanKiss 5y ago

well if you combine the title with the contents you reach the conclusion that python is not a programing language. fair enough.

pphysch 5y ago

It's the usual trade-off of individual programmer freedom/expressiveness vs. rigid & predictable standards that enable productivity at scale. Just because you can program a new feature quickly doesn't mean it is engineered well for the long-term & larger scales.

woah 5y ago

I thought this was going to be satire, but no, it seems that he is completely serious.

saila 5y ago
Is the premise unreasonable? Of course, it depends on what you're doing--there are certainly use cases where you know Python or a similar language wouldn't be appropriate (for performance or other reasons)--but it seems to me that there are plenty of scenarios where using a "faster" language up front isn't warranted, and you may never need to switch.

hsn915 5y ago

The fallacy of premature optimization rears its ugly head again. When Knuth wrote that quote, he meant something very specific with the word optimization: low level micro optimizations. Spending a lot of time trying to get every little ounce of performance from every little CPU instruction. High level reasonable design decisions are not "optimizations" in this sense at all. You should think of them as non-pessimization. Choosing Python or another slow language is premature pessimization. You just make your program slower for no reason. So choosing a language based on how programs written in it perform is simply non-pessimization. If you are interested in an expansion on this idea, checkout Casey Muratori's lecture about philosophies of optimization: https://www.youtube.com/watch?v=pgoetgxecw8 https://www.youtube.com/watch?v=pgoetgxecw8 The article makes claims about software being "fast enough" and not needing further optimization. But what you call "fast enough" is probably 10000 times slower than what it could be. I'm not joking. People who program in slow languages have a really skewed perception about performance. If a page takes 5 seconds to load, they don't see that as a problem. If a server takes 500ms to respond to a request, they don't see that as a problem. This is simply unacceptable.

dragonwriter 5y ago
> High level reasonable design decisions Viewing language choice as a high level design decision for a project of nontrivial scale is a symptom of bad high-level design decisions.
flohofwoe 5y ago
OTH not choosing Python for tasks where the language performance really doesn't matter at all (for instance shell scripting stuff) would also be futile. It's simply a matter of using the right tool for a specific job (and in Python's case, that 'tool' isn't even the language, but the batteries-included stdlib).
- majou 5y ago
  OK but shell is cuter than python.
- hsn915 5y ago

mumblemumble 5y ago

A case study from my own personal experience: We generally do everything in Java. Python is avoided because of all the usual complaints - dynamically typed, not fast enough, GIL, etc. So, as a PoC, I decided to try rewriting one of our services in Python. And, compared to the Java one, it is: Cheaper to write and maintain. About 1/10 as many SLOC. About 1/20 as many person-hours. Has better static type checks. For example, Java's type checker cannot statically verify for null safety. The Python one I chose does do that. Note that, since I did choose to put type hints on everything, the productivity boost in question cannot be attributed to dynamic typing. (That said, not all 3rd-party libraries have type hints, so, if you want to type check everything, you may have some yak shaving to do.) Uses less RAM. About 1/2 as much. Is faster. Admittedly I'm leaning on numpy, Cython, and friends for this. It may well be much slower for projects where that is not possible. But still, I think that the point about premature operation stands in this case. (Disclaimer: I also have a colleague at $FAMOUS_COMPANY who tells me they are moving off of Python because they found the opposite of the above in many cases. Though I personally suspect that a mitigating factor is that their profit margins and scale are large enough that all the coefficients in their cost/benefit formula are wildly different from the norm.)

killingtime74 5y ago
Kotlin Scala does all these and interface with Java to boot
TheDudeMan 5y ago
> Java's type checker cannot statically verify for null safety. Check out NullAway. > Uses less RAM. About 1/2 as much. Were you on a modern JVM?
- mumblemumble 5y ago
  It's not just null. For example, Java much more frequently leaves you in a situation where you need to cast to/from `Object`. In Python, `Any` type hints can usually be avoided, because of union types.

GuB-42 5y ago

Python is not just slow, it also tends to break down for large projects, like all dynamic languages. It is not an absolute, you can do it, but the larger the code base, the more you need strong guarantees over flexibility, and the less relevant languages like Python tend to become. Prototyping in Python is a viable strategy. Personally I tend to use Perl for that, because I am comfortable with it and I find it better adapted to quick prototyping than Python. I then rewrite in another language, usually C++, with more care about data structures and optimization. The author strategy to do everything in Python than optimize the slow parts can make sense, though I tend to prefer the opposite: white your framework/engine in a static language (like C++) and embed an interpreter (Python, LUA, etc...), as it is common in game dev. But all that are language decisions! Whatever you do, there are going to be consequences. Choosing Python with C/C++/Rust optimizations is not bad, but it is a rather strong opinion, not the obvious default choice the author makes it.

hsn915 5y ago
> it also tends to break down for large projects You also end up spending a lot of time and resources in the later stages of the project trying to work around the problems: hire more people to work on the code base, spend a lot of time investigating optimization strategies, hire a devops team to turn the program into a distributed system with maybe hundreds of instances running on CloudProvider (TM)
- api 5y ago
  > turn the program into a distributed system with maybe hundreds of instances running on CloudProvider (TM) ... and give tens to hundreds of thousands a month to cloud providers when you could spend 1/100th if it were written in Go.
  - hsn915 5y ago
    You could even save the costs by buying your own server. I doubt that you can't buy a very decent server hardware for no more than $10000. Some places pay more than that per month to Amazon.

dragonwriter 5y ago

I agree with the broad concept (with the caveat that there are exceptions to all generalizations), but note that “Python” can generally be replaced with “whatever general purpose language you are comfortable and familiar with that doesn't have any clear insurmountable barriers like ‘is not available for the desired target platform’.”

everyone 5y ago

Just use what you know. Something you've used before.

letwhile 5y ago

While I see a point in this article, I personally run into performance problems with python at a very early stage nearly every time. The overhead of starting to use some ffi is huge. Suddenly you have two projects, with two different build systems etc. Way faster to just use a performant language. Besides this, dynamic typing gets frustrating quickly, as the loc rise, and distribution is not very fun. I love the language, but won't use it anymore for any "serious" project.

eptcyka 5y ago

I really don't find writing in Python to be that much more productive than using many other modern language.

xg15 5y ago

This sounds like the "Java can be fast" arguments from two decades or the "JavaScript can be fast" arguments from one decade ago, now with the next language. I mean, sure I can prototype in python, then put a lot of additional effort into bringing performance of the bits I absolutely need fast close to native code performance. Or I could code everything in C or Rust and get everything in native code from the get-go.

deleted 5y ago

[deleted]