13 ms·
Python keeps growing in number of users because it’s easy to get started, has libraries to load basically any data, and to perform any task. It’s frequently the
by pletnes 4y ago
Python keeps growing in number of users because it’s easy to get started, has libraries to load basically any data, and to perform any task. It’s frequently the second best language but it’s the second best language for anything.
By the time a python programmer has «graduated» to learning a second language, exponential growth has created a bunch of new python programmers, most of which don’t consider themselves programmers.
There are more non-programmers in this world, and they don’t care - or know about - concurrency, memory efficiency, L2 cache misses due to pointer chasing. These people all use python. This seems to be a perspective missing from most hackernews discussions, where people work on high performance Big corp big data web scale systems.
- SushiHippie 4y agoYeah, big corp big data web scale systems use python too.
- xaitv 4y agoThere's also the argument that at a certain scale the time of a developer is simply more expensive than time on a server. If I write something in C++ that does a task in 1 second and it takes me 2 days to write, and I write the same thing in Python that takes 2 seconds but I can write it in 1 day, the 1 day of extra dev time might just pay for throwing a more high performance server against it and calling it a day. And then I don't even take the fact that a lot of applications are mostly waiting for database queries into consideration, nor maintainability of the code and the fact that high performance servers get cheaper over time. If you work at some big corp where this would mean thousands of high performance servers that's simply not worth it, but in small/medium sized companies it usually is.
- eska 4y agoIf Python was merely twice as slow then I could agree with you.
- bombolo 4y agoNot all code needs to process terabytes of data. I have code running that reads ~20 bytes, checks the internal status on an hashmap and flips a bit. Would it be faster in C? Of course. Would it have taken me much longer to write to achieve absolutely no benefit? Yes.
- dagw 4y agoRealistically something that takes 1 second in C++ will take 10 seconds (if you write efficient python and lean heavily on fast libraries) to 10 minutes in python. But the rest of your point stands
- joenot443 4y agoDamn! Is the rule of thumb really a 10x performance hit between Python/C++? I don’t doubt you’re correct, I’m just thinking of all the unnecessary cycles I put my poor CPU through.
- benji-york 4y agoAs a long-time Python lover, yes that's a decent rule of thumb.
- vgatherps 4y agoOutside cases where Python is used as a thin wrapper around some C library (simple networking code, numpy, etc) 10x is frankly quite conservative. Depending on the problem space and how aggressively you optimize, it's easily multiple orders of magnitude.
- DiogenesKynikos 4y agoThose cases are about 95% of scientific programming. This is the first line in most scientific code: import numpy
- intelVISA 4y agoFFI into lean C isn't some perf panacea either, beyond the overhead you're also depriving yourself of interprocedural optimization and other Good Things from the native space.
- sva_ 4y agoIt really depends on what you're doing, but I don't think it is generally accurate. What slows Python down is generally the "everything is an object" attitude of the interpreter. I.e. you call a function, the interpreter has to first create an object of the thing you're calling. In C++, due to zero-cost abstractions, this usually just boils down to a CALL instruction preceded by a bunch of PUSH instructions in assembly, based on the number of parameters (and call convention). This is of course a lot faster than running through the abstractions of creating some Python object.
- mharig 4y agoSpeeding up the time critical parts with Cython or Numba or ... is rather easy.
- amelius 4y agoThere are also programmers who are tired of chasing pointers and simply want to get stuff done. E.g. people who once wrote "robust" code in Rust but were "outcompeted" left and right by coworkers who churn out shiny new things at 10x the speed.
- a4a4a4a4 4y agoOr there are programmers who write both. Something that I want to write once, have run on several different platforms, handle multi-threading nicely, and never have to think about again? Rust. Writing something to read in some data to unblock an ML engineer or make plots for management? Definitely not Rust, probably python. Then you can also churn out things at 10x the speed, but by writing the tricky parts in something other than python, you don't get dragged back down by old projects rearing their ugly heads, so you outpace the python-only colleagues in the long-term.
- jamincan 4y agoProgramming is secondary to my primary duties and only a means for me to get other things done. I'm in constant tension between using Python and Rust. With Python I can get things up and going very quickly with little boilerplate, but I find that I'm often stumbling on edge cases that I have to debug after the fact and that these instances necessarily happen exactly when I'm focused on another task. I also find that packaging for other users is a major headache. With Rust, the development time is much higher for me, but I appreciate being able to use the type-system to enforce business logic and therefore find that I rarely have to return to debug some issue once I have it going. It's a tough trade-off for me, because I appreciate the velocity of Python, but Rust likely saves me more time overall.
- temp2022account 4y ago> coworkers who churn out shiny new things at 10x the speed Sounds like a classic web-dev perspective, my customers hate when we ship broken tools because it ruins their work, new feature velocity be dammned. We love our borrow checker because initially you run at 0.5x velocity but post-25kSLOC you get to run at 2x velocity, which continues to mystify managers worldwide.
- traverseda 4y ago>and they don’t care - or know about - concurrency, memory efficiency, L2 cache misses due to pointer chasing. Also if I (a programmer) want to write really really fast code I'm probably reaching for tools like tensorflow, numpy, or jax. So there's not much incentive for me to switch to a more efficient language when as near as I can tell the best tooling for dealing with SIMD or weird gpu bullshit seems to be being created for python developers. If you want to write fast code do it in c/rust/whatever, if you want to write really fast code do it in python with <some-ML-library>. For a very specific definition of the word "fast" at least.
- FpUser 4y ago>"Also if I (a programmer) want to write really really fast code I'm probably reaching for tools like tensorflow, numpy, or jax." Very limited view which ignores way too many areas.
- traverseda 4y agoSure, examples?
- vgatherps 4y agoYour OS, the linear algebra libraries themselves, much of the user-facing software that you use (latency sensitive rather than throughput sensitive), image/video encoding/decoding, most of the language runtimes that you use, high volume webservers, high volume data processing (where your data is not already some nice flat list of numbers you're operating on with tensor operations), for some examples. Really, for almost any X, somebody somewhere has to do X with strict performance requirements (or at very large scale, so better perf == savings) Most of these python libraries are only fast for relatively large and relatively standard operations in the first place. If you have a lot of small/weird computations, they come with a ton of overhead. I've personally had to write my own fast linear algebra libraries since our hot loop was a sort of modified tropical algebra once.
- 4y ago
- RcouF1uZ4gsC 4y agoYour comment is super interesting because it suggests Python has evolved in a direction opposite to the Python Paradox - http://www.paulgraham.com/pypar.html http://www.paulgraham.com/pypar.html Whereas before you could get smarter programmers using Python, now because of the exponential growth of Python, the median Python programmer is likely someone with little or no software engineering or computer architecture background who is basically just gluing together a lot of libraries.
- kittiepryde 4y agoNeat observation. I wasn't doing much programming in 2004, but, I'm guessing 2004 Python would be like today's Rust. People learn it because they love it.
- RcouF1uZ4gsC 4y agoI think more so Rust than even Python on 2004 since Rust has a pretty steep learning curve and does require a non-trivial amount of dedication to learning it.
- analog31 4y agoPerhaps today its not smart programmers per se, but smart people who are interested in learning to program. The libraries are the killer feature for me.
- moffkalast 4y ago> it’s the second best language for anything Anything that doesn't require high performance that is. Is there any 3D game engine for python yet? I guess Godot has gdscript which is 90% python by syntax, but that doesn't quite count I think.
- loudmax 4y agoYou won't get high performance out of Python directly, but there are a lot of Python libraries that use C or a powerful low level language underneath. The heavy lifting in so much of machine learning is CUDA, but most people involved in ML are writing Python.
- moffkalast 4y agoSure, but what's not really python per se. One could also call C++ libraries from java via JNI and pretend java is super fast. If people write program logic in python it will run at python speeds. Otherwise you're not really writing python, like nobody says some linux native program is bash because it happens to be launched from a bash script.
- martopix 4y ago> Sure, but what's not really python per se. But that's exactly the strength of Python: it's an interface language. It's meant to make pretty sophisticated things like CUDA easy for everyone.
- kaba0 4y agoJava is super fast though, it almost never uses JNI as it doesn’t need it as opposed to Python. It uses JNI for integrating with the C world (e.g. opengl bindings).
- yamtaddle 4y ago> Sure, but what's not really python per se. One could also call C++ libraries from java via JNI and pretend java is super fast. But that's how every scripting language obtains good-not-just-decent performance. A strong culture of dropping down to C for any halfway-important library is why PHP's so hard to beat in real-world use, speed-wise (whatever its other shortcomings).
- benrutter 4y agoI completely agree - but you say that like it's a bad thing. I work as a developer alongside data scientists, who might have strong knowledge of statistics or machine learning frameworks rather than traditional programming chops. For the most part they don't need to know about concurrency, memory efficiency etc, because they're using a library where those issues have been abstracted away. I think that's what makes python ideal - it's interoperability with other languages and library ecosystem means less technical people can produce good, efficient work without having to take on a whole bunch of the footguns that would come from working directly in a language like c++ or Rust.
- roflyear 4y agoMost programmers don't actually need to know about that stuff, either. And most programmers who do need to know about that stuff, don't know about it.
- _aavaa_ 4y agoBut this is a false dichotomy. The space of options isn't C++/Rust or Python. There are languages which attempt to give the best of both worlds, e.g. Julia. > they're using a library where those issues have been abstracted away. I work in Python, and while libraries like numpy have certainly abstracted away some of those issues, there's still so much performance left of the table because Python is still Python.
- ofrzeta 4y ago
- Zingler 4y ago> It’s frequently the second best language but it’s the second best language for anything. This myth wasn't even true many years ago, it certainly isn't true today. You can build a mobile app, game, distributed systems, OS, GUI, Web frontend, "realtime" systems, etc in Python, but it is a weak choice for most of those things (and many others) let alone the second best option.
- toastal 4y agoWhen I switched from PHP to Python years ago I had the same feeling as the OP, then it became the third best, then the fourth, then situational when object-orientation makes sense, then for just scripting, and now... unsure beyond a personal developer comfort/productivity preference. TUIs and GUIs built on Python on my machine seem to be the first things to have issues during system upgrades because of the package management situation.
- brookst 4y agoThe saying does not mean that in a rigorous evaluation Python would be second best out of all programming ecosystems for all problems. The saying means that for any given problem, there is a better choice, but second best is the language you know which has all of the tools to get the job done, so the answer is probably just a bunch of pip installs, imports, and glue code. It’s kind of like “the best camera is the one you have with you” — it’s a play on the differing definitions of “best” to highlight the value of feasibility over technical perfection.
- deleted 4y ago[deleted]
- Yoric 4y agoI fully agree with the description. What worries me, though, is that the features that make Python quite good at prototyping make it rather bad at auditing for safety and security. And we live in a world in which production code is prototyping code, which means that Python code that should have remained a quick experiment – and more often than not, written by people who are not that good at Python or don't care about code quality – ends up powering safety/security-critical infrastructures. Cue in the thousands of developer-hours debugging or attempting to scale code that is hostile to the task. I would claim that the same applies to JavaScript/Node, btw.
- awkward 4y agoWe still live in a world where many outward facing networked applications are written in C. Dynamic languages with safe strings are far from the floor for securable tools.
- Yoric 4y agoThat is true. However, I hope that these C applications are written by people who are really good at C. I know that some of these Python applications are written by people who discovered the language as they deployed into production.
- KyeRussell 4y agoThat’s a measure of programming prowess, not the actual security concern at hand. If the masterful C developer still insists on using a language that has so many footguns and a weird culture of developers pretending that they’re more capable than they are, then their C mastery could very well’ve not been worth much against someone throwing something together in Python, which will at the very least immediately bypass the vast majority of vulnerabilities found in C code. Plus, my experience with such software is that the sort of higher level vulnerabilities that you’d still see in Python code aren’t ones that the C developer has necessarily dealt with.
- deleted 4y ago
- throwaway894345 4y agoI’ve been a Python developer for 15 years, and Python might have been the second best language for anything when I started my career, but there are so many better options for just about any domain except maybe data science. Basically for any domain that involves running code in a production environment (as opposed to iterating in a Jupiter notebook) in which you care about reliability or performance or developer velocity, Python is going to be a pretty big liability (maybe it will be manageable if you’re just building a CRUD app atop a database). Common pain points include performance (no you can’t just multiprocess or numpy your way out of performance problems), packaging/deployment, and even setting up development environments that are reasonably representative of a production environment (this depends a lot on how you deploy to production—I’m sure lots of people have solved this for their production environment).
- hot_gril 4y agoPython isn't a joke either. I'm a full-on programmer who started with C and branched out to several other languages, and I'd still pick Python for a lot of new tasks, even things that aren't little scripts. Or NodeJS, which has similar properties but has particular advantages for web backends.