9 ms·
I always wondered how Python can be one of the world's most popular languages without anyone (company) stepping up and make the runtime as fast as modern JavaSc
by cryptos 3y ago
I always wondered how Python can be one of the world's most popular languages without anyone (company) stepping up and make the runtime as fast as modern JavaScript runtimes.
- tomwphillips 3y agoBecause enough users find the performance sufficient.
- rfoo 3y agoBecause the reason why Python is one of the world's most popular language (a large set of scientific computing C extensions) is bound to every implementation details of the interpreter itself.
- fer 3y agoEasy and the number-crunching libs are optimized away in (generally) C.
- PartiallyTyped 3y agoand FORTRAN.
- est31 3y agoPython is already fast where it matters: often, it is just used to integrate existing C/C++ libraries like numpy or pytorch. It is more an integration language than one where you write your heavy algorithms in. For JS, during the time that it received its JITs, there was no cross platform native code equivalent like wasm yet. JS had to compete with plugins written in C/C++ however. There was also competition between browser vendors, which gave the period the name "browser wars". Nowadays at least, the speed improvements for the end user thanks to JIT aren't also that great, Apple provides a mode to turn off JIT entirely for security.
- ephimetheus 3y agoI think usually the term “browser wars” refers to the time when Netscape and Microsoft were struggling for dominance, which concluded in 2001. JavaScript JITs only emerged around 2008 with SpiderMonkey’s TraceMonkey, JavaScriptCore’s SquirrelFish Extreme, and V8’s original JIT.
- lifthrasiir 3y agoThere were multiple browser wars, otherwise you wouldn't need -s there ;-)
- nyanpasu64 3y agoHaving recently implemented parallel image rendering in corrscope (https://github.com/corrscope/corrscope/pull/450 https://github.com/corrscope/corrscope/pull/450), I can say that friends don't let friends write performance-critical code in Python. Depending on prebuilt C++ libraries hampers flexibility (eg. you can't customize the memory management or rasterization pipeline of matplotlib). Python's GIL inhibits parallelism within a process, and the workaround of multiprocessing and shared memory is awkward, has inconsistencies between platforms, and loses performance (you can't get matplotlib to render directly to an inter-process shared memory buffer, and the alternative of copying data from matplotlib's framebuffer to shared memory wastes CPU time). Additionally a lot of the libraries/ecosystem around shared memory (https://docs.python.org/3/library/multiprocessing.shared_memory.html https://docs.python.org/3/library/multiprocessing.shared_mem...) seems poorly conceived. If you pre-open shared memory in a ProcessPoolExecutor's initializer functions, you can't close them when the worker process exits (which might be fine, nobody knows!), but if you instead open and close a shared memory segment on every executor job, it measurably reduces performance, presumably from memory mapping overhead or TLB/page table thrashing.
- xiphias2 3y agoBillions of dollars of product decisions use JS benchmark speed as one of the standard benchmarks to base buying decision on (for a good reason). For machine learning speed compiling to the right CUDA / OpenCL kernel is much more crucial, so there's where the money goes.
- CJefferson 3y agoA big part of what made Python so successful was how easy it was to extend with C modules. It turns out to be very hard to JIT Python without breaking these, and most people don’t want a Python that doesn’t support C extension modules. The JavaScript VMs often break their extensions APIs for speed, but their users are more used to this.
- toyg 3y agoJS doesn't really have the tradition of external modules that Python has, for a long time it only really existed inside the browser.
- amelius 3y agoOn the other hand, rewriting the C modules and adapting them to a different C API is very straightforward after you've done 1 or 2 of such modules. Perhaps it's even something that could be done by training an LLM like Copilot.
- Redoubts 3y agoThat's breakage you'd have to tread carefully on; and given the 2to3 experience, there would have to be immediate reward to entice people to undertake the conversion. No one's interested in even minor code breakage for minor short-term gain.
- Pxtl 3y agoWhich is why I'm shocked that Python's big "we're breaking backwards compatibility" release (Python 3) was mostly just for Unicode strings. It seems like the C API and the various __builtins__ introspection API thingies should've been the real focus on breaking backwards compatibility so that Python would have a better future for improvements like this.
- AndrewDucker 3y agoAlways interested in replies to this kind of comment, which basically boil down to "Python is so slow that we have to write any important code in C. And this is somehow a good thing." I mean, it's great that you can write some of your code in C. But wouldn't it be great if you could just write your libraries in Python and have them still be really fast?
- dagw 3y agoBut wouldn't it be great if you could just write your libraries in Python Everybody obviously wants that. The question is are you willing to lose what you have in order to hopefully, eventually, get there. If Python 3 development stopped and Python 4 came out tomorrow and was 5x faster than python 3 and a promise of being 50-100x faster in the future, but you have to rewrite all the libraries that use the C API, it would probably be DOA and kill python. People who want a faster 'almost python' already have several options to choose from, none of which are popular. Or they use Julia.
- AndrewDucker 3y agoWhy are you assuming that they'd have to rewrite all of their libraries? I don't see anything in the article that says that.
- dagw 3y agoThe reason this approach is so much slower than some of the other 'fast' pythons out there that have come before is that they are making sure you don't have to rewrite a bunch of existing libraries. That is the problem with all the fast python implementations that have come before. Yes, they're faster than 'normal' python in many benchmarks, but they don't support the entire current ecosystem. For example Instagram's python implementation is blazing fast for doing exactly what Instagram is using python for, but is probably completely useless for what I'm using python for.
- AndrewDucker 3y ago
- yellowstuff 3y agoThere have been several attempts. For example, Google tried to introduce a JIT in 2011 with a project named Unladen Swallow, but that ended up getting abandoned.
- albertzeyer 3y agoIn lots of applications, all the computations already happen inside native libraries, e.g. Numpy, PyTorch, TensorFlow, JAX etc. And if you have a complicate computation graph, there are already JITs on this level, based on Python code, e.g. see torch.compile, or TF XLA (done by default via tf.function), JAX, etc. It's also important to do JIT on this level, to really be able to fuse CUDA ops, etc. A generic Python JIT probably cannot really do this, as this is CUDA specific, or TPU specific, etc.
- dagw 3y agoanyone (company) stepping up and make the runtime as fast as modern JavaScript runtimes. There are a lot of faster python runtimes out there. Both Google and Instagram/Meta have done a lot of work on this, mostly to solve internal problems they've been having with python performance. Microsoft has also done work on parallel python. There's PyPy and Pythran and no doubt several others. However none of these attempts have managed to be 100% compatible with the current CPython (and more importantly the CPython C API), so they haven't been considered as replacements. JavaScript had the huge advantage that there was very little mission critical legacy JavaScript code around they had to take into consideration, and no C libraries that they had to stay compatible with. Meaning that modern JavaScript runtime teams could more or less start from scratch. Also the JavaScript world at the time were a lot more OK with different JavaScript runtimes not being 100% compatible with each other. If you 'just' want a faster python runtime that supports most of python and many existing libraries, but are OK with having to rewrite some your existing python code or third party libraries to make it work on that runtime, then there are several to choose from.
- skriticos2 3y agoJS also had the major advantage of being sandboxed by design, so they could work from there. Most of the technical legacy centered around syntax backwards compatibility, but it's all isolated - so much easier to optimize. Python with it's C API basically gives you the keys to the kingdom on a machine code level. Modifying something that has an API to connect to essentially anything is not an easy proposition. Of course, it has the advantage that you can make Python faster by performance analysis and moving the expensive parts to optimized C code, if you have the resources.
- Pxtl 3y agoNode.js and Python 3 came out at around the same time. Python had their chance to tell all the "mission critical legacy code" that it was time to make hard changes.
- dagw 3y agoAs much as I would have loved to see some more 'extreme' improvements to python, given how the python community reacted to the relatively minor changes that python 3 brought, anything more extreme would very likely have caused a Perl 6 style situation and quite possibly have killed the language.
- JodieBenitez 3y agoBecause it's already fast enough for most of us ? Anecdote, but I've had my share of slow things in Javascript that are not slow in Python. Try to generate a SHA256 checksum for a big file in the browser... Good to see progress anyways.
- jampekka 3y agoPython's SHA256 is written in C. And I'd quess Web Crypto API for JS is in the same ballbark. SHA256 in pure Python would be unusably slow. In Javascript it would be at least usably slow. Javascript is fast. Browsers are fast.
- Scarblac 3y agoThe point of Python is quickly integrating a very wide range of fast libraries written in other languages though, you can't ignore that performance just because it's not written in Python.
- JodieBenitez 3y agoHave you tried to generate a SHA256 checksum for a file in the browser, no matter what crypto lib or api is available to you ? Have you tried to generate it using Python standard lib ? I did, and doing it in the browser was so bad that it was unusable. I suspect that it's not the crypto that's slow but the file reading. But anyway... > SHA256 in pure Python would be unusably slow None would do that because: > Python's SHA256 is written in C Hence why comparing "pure python" to "pure javascript" is mostly irrelevant for most day to day tasks, like most benchmarks. > Javascript is fast. Browsers are fast. Well, no they were not for my use case. Browsers are really slow at generating file checksums.
- adastra22 3y agoThe Pytthon standard lib calls out to hand optimized assembly language versions of the crypto algos. It is of no relevance to a JIT-vs-interpreted debate.
- el_oni 3y agoI think the thing with python is that it's always been "fast enough" and if not you can always reach out to natively implemented modules. On the flipside javascript was the main language embedded in web browsers. There has been a lot of competition to make browsers fast. Nowadays there are 3 main JS engines, V8 backed by google, JavaScriptCore backed by apple, and spidermonkey backed by mozilla. If python had been the language embedded into web browsers, then maybe we would see 3 competing python engines with crazy performance. The alternative interpreters for python have always been a bit more niche than Cpython, but now that Guido works at microsoft there has been a bit more of a push to make it faster
- PartiallyTyped 3y agoMeta has actually been doing that — helping improve python's speed — with things like [1,2] [1] https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/ [2] https://news.ycombinator.com/item?id=36643670 https://news.ycombinator.com/item?id=36643670
- tgv 3y agoTeaching. So many colleges/unis I know teach "Introduction to Programming" with Python these days, especially to non-CS students/pupils.
- bigfishrunning 3y agoI think python is very well suited to people who do computation in Excel spreadsheets. For actual CS students, I'd rather see something like scheme be a first language (but maybe I'm just an old person)
- hot_gril 3y agoThey do both Python and Scheme in the same Berkeley intro to CS class. But I think the point of Scheme is more to expand students' thinking with a very different language. The CS fundamentals are still covered more in the Python part of the course.
- VagabundoP 3y agoIts even in Excel nowadays!!
- hot_gril 3y agoand in Postgres UDFs
- fractalb 3y agoI still scratch my head why it’s not installed by default on Windows.
- Yasuraka 3y agoYou might want to checkout Mojo, which is not a runtime but a different language, but also designed to be a superset of Python. Beware though that it's not yet open source, which is slated for this Q1 https://docs.modular.com/mojo/manual/ https://docs.modular.com/mojo/manual/ edit: The main point I forgot to mention - it aims to compete with "low-level" languages like C and Rust in performance
- FergusArgyll 3y agoBecause it doesn't useGrossCamelCaseAsOften
- IshKebab 3y agoTwo reasons: 1. Javascript is a less dynamic language than Python and numbers are all float64 which makes it a lot easier to make fast. 2. If you want to run fast code on the web you only have one option: make Javascript faster. (Ok we have WASM now but that didn't exist at the time of the Javascript Speed wars.) If you want to run fast code on your desktop you have a MUCH easier option: don't use Python.
- soulbadguy 3y ago> Javascript is a less dynamic language than Python I have seen this mentioned multiple times, someone as a good reference explaining what makes python more dynamic than JS ?
- IshKebab 3y agoIt just has many features that can be overridden with Python code so they can have any behaviour at runtime. For example when you access an object attribute it can literally do anything: https://docs.python.org/3/reference/datamodel.html#customizing-attribute-access https://docs.python.org/3/reference/datamodel.html#customizi... You could probably optimistically optimise some code, assuming it doesn't use any of the dynamic features of Python. You're going to get crazy performance cliffs though.
- bjackman 3y agoJavaScript has to be fast because its users were traditionally captive on the platform (it was the only language in the browser). Python's users can always swap out performance critical components to another language. So Python development delivered more when it focussed on improving strengths rather than mitigating weaknesses. In a way, Python being slow is just a sign of a healthy platform ecosystem allowing comparative advantages to shine.
- hot_gril 3y agoNew runtimes like NodeJS have expanded JS beyond web, and JS's syntax has improved the past several years. But before that happened, Python on its own was way easier for non-web scripts, web servers, and math/science/ML/etc. Optimized native libs and ecosystems for those things got built a lot earlier around Python, in some cases before NodeJS even existed. Python's syntax is still nicer for mathy stuff, to the point where I'd go into job coding interviews using Python despite having used more JS lately. And I'm comparing to JS because it's the closest thing, while others like Java are/were far more cumbersome for these uses.
- deleted 3y ago[deleted]