11 ms·
I have to say that my first reaction was: "maybe you shouldn't use python for this, then". If you are using a language in a way that it gets worse in subsequen
by rossdavidh 9y ago
I have to say that my first reaction was: "maybe you shouldn't use python for this, then". If you are using a language in a way that it gets worse in subsequent versions, that's a good sign that they're optimizing for something other than what you care about.
The programming language R does not, as I understand it, optimize for speed, because they are optimizing for ease of exploratory data analysis. R is growing quite rapidly. So is python, actually. It doesn't mean that either one is good at everything, and it's probably the case that both are growing because they don't try to be good at everything. A good toolbox is better than a multi-tool.
- MBCook 9y agoThe supposed attitude of the python developers about startup time works against the popular niches Python is supposed to be such a great fit for. Little scripts, glue, short run applications. That’s a problem if that’s an area python wants to compete in.
- carapace 9y agoYep. Tried to use a Raspberry Pi as my main system for a while and one of the pain points was slooooow startup of Python. As a Python fan I was embarrassed.
- thanatropism 9y agoI might be biased because I'm from the hordes that are moving from Stata and Matlab to Python (but then there are the hordes attracted to data analysis now), but that was never really Python's strong suit, nor its target market. I mean, I was always into little scripts, but I used Tcl and then Perl.
- eesmith 9y agoBack in the 1990s, Python was promoted as a web programming language. This was back in the days when everyone used CGIs. Python came with an cgi module, while in Perl you had to download cgi-lib.pl. I even helped maintain a Python web application that was all CGI-based. So I can assure you that at one point Python was trying to be in the "short run applications" space. They may have given up since then, but that's a different issue. As for me, I do write little scripts in Python. I don't like how most of my run time is spent waiting for Python to get ready. What I really don't like is using NumPy. I tend to re-implement features I want rather than reach for NumPy because that 0.2s import time irks me so much. And it's because the NumPy developers want people to do "import numpy; numpy.reach.into.a.deep.package", so they import most of its submodules. They used to also eval() some code at import, causing even more overhead. I don't know if that's gone away.
- lmm 9y agoBoth Tcl and Perl are dead languages walking these days, and it's Python that's displaced them. It absolutely competes in that market.
- blacksqr 9y agoMarkets are a funny thing. Both "dead" languages are thread safe and can easily run separate interpreters per thread. See: "THREADS DONE RIGHT... WITH TCL" https://www.activestate.com/blog/2016/09/threads-done-right-tcl https://www.activestate.com/blog/2016/09/threads-done-right-...
- catach 9y agoI do have to wonder if Tcl would have gained significantly greater mindshare if the syntax had been more Algol-like.
- lmm 9y ago> Markets are a funny thing. Both "dead" languages are thread safe and can easily run separate interpreters per thread. Perl threading is officially recommended against IIRC? In either case, "threads" in either of them don't share memory (except explicitly and manually), at which point what you have is multiprocessing by a different name.
- lizmat 9y agoIn Perl 5 threading things are only "shared" semantically. In reality, there is an extra interpreter running where the "shared" variables live, and any value fetches and stores are handled with the `tie` interface (see http://www.perlmonks.org/?node_id=288022 http://www.perlmonks.org/?node_id=288022 for more information). In Perl 6 on the other hand, everything is always shared and you can use high level constructs such as atomic increments, supplies, taps, `react` and `whenever` so you don't have to think about any (dead)locking issues as a developer.
- blacksqr 9y agoTcl has an "easy" threading mode where each thread runs its own interpreter, and a "hold my beer" mode where you can spawn threads within a single interpreter. Tcl also has synthetic channels, so you can stay in easy mode but open a bi-directional read/write channel with the two ends in separate threads, with readable/writable events assigned, so you can do automatic event-driven information sharing between threads. I don't know any other language that gives you options like that.
- alayne 9y agoThe linked post is about Python startup being a problem with thousands of invocations. Is Python startup really a problem for the niches you mention, or is it a problem in some extreme edge cases? I would argue this is the latter and perhaps signals that an architecture change for the build or tests would be best. I have been using Python for small scripts for 20+ years and haven't had this issue. The JVM on the other hand was historically slow to start.
- PeterisP 9y agoIf you need to run thousands of scripts, do you need to (re-)start Python for each script? IMHO what needs to be done for this problem is not faster startup, but a way to avoid startup by implementing a feature where you can keep a single Python "machine" in memory that can make a "soft reset" to execute a fresh script.
- pas 9y agoYep. Even PHP solved this :) That said PHP startup (parsing, because there's no on disk bytecode cache like .pyc - though there's an in-memory one [OPcache], as somewhat expected from a server thingie) was always quite fast, and it got a bit faster in php7: https://wiki.php.net/rfc/abstract_syntax_tree#impact_on_performance_and_memory_usage https://wiki.php.net/rfc/abstract_syntax_tree#impact_on_perf...
- jMyles 9y agoI don't particularly agreed about this being what "Python is supposed to be such a great fit for." I've been to quite a few PyCons and never heard anyone espousing this view, but I'm open to the possibility that I have missed it. Can you link me to a piece of media that you think persuasively makes the case that this is what Python is supposed to be for?
- busterarm 9y agoPython is really only the target for those because someone lied to all of the systems folk and told them that Ruby was too slow. (The previous wave of infrastructure management tools seemed to all be written in Ruby and nowadays it's Python or Go.) That and python is one of the "official" languages at Google and everyone wants to be Google, right? Meanwhile, Ruby is making great strides in performance and even has JIT coming in 2.6.
- marcosdumay 9y agoPython is not optimized for small glue code at all. The fact that it is the sanest language for use in that niche speaks much more about the ecosystem than about Python. Python seems to be mainly optimized for web servers, scientific computing and machine learning tasks. None of those care about startup time.
- indygreg2 9y ago(I authored the linked post) While the "maybe you shouldn't use Python" comment could be construed as trolling to some, there is definite truth to your line of reasoning and I agree with comment. I absolutely love Python as a programming language for the space it is in. But as someone who needs to think long term about maintaining large projects with lifetimes measured in potentially decades, Python has a few key weaknesses that make it really difficult for me to continue justify using it for such projects. Startup time is one. The GIL is the other large one (not being able to achieve linear speedups on CPU-bound code in 2018 with Moore's Law dead is unacceptable). General performance disadvantages can be adequately addressed with PyPy, JITs, Cython, etc. Problems scaling large code bases using a dynamic language can be mitigated with typing and better tools. Python can be very competitive against typed systems languages. But if it fails to address its shortcomings, I think more and more people will choose Rust, Go, Java, C/C++, etc for large scale, long time horizon projects. This will [further] relegate Python to be viewed as a "toy" language by more serious developers, which is obviously not good for the Python ecosystem. So I think "maybe you shouldn't use Python for this, then" is a very accurate statement/critique.
- rossdavidh 9y agoSorry, I didn't mean it to be trolling, I just meant it more or less literally. If Rust (for example) gets used for things like Mercurial and Mozilla, is that bad? I'm not saying Python shouldn't care, if it could improve the startup time without sacrificing other things. But presumably the transition from py2 to py3 was not intending to make things slower, it was intending to solve other problems. There are almost always tradeoffs. Even the mercurial folks quoted in the article said that the things py3 solved were not what they needed. That's a good indicator that Python is not the right language (anymore) for what they're doing. I am primarily a Python programmer, but if Rust, Go, etc. take over as the language of choice in certain cases, I don't think that's a bad thing. Which doesn't mean one shouldn't write an article to highlight this cost of not having short startup time, just in case this cost wasn't understood by Guido, et al. But my guess (and it's only a guess), is that it was.
- m_mueller 9y agoRegarding the GIL I‘ve always wondered about Jython but never gotten around to trying it. What are the drawbacks of running it on a JVM to get true multithreading? Having to properly sync the threads like in other environments without global locks?
- pc2g4d 9y agoI think this is why Mercurial is switching (largely) to Rust: https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan