10 ms·
Implementation plan for speeding up CPython
- coldtea 6y agoHaven't seen this discussed on HN, and seems like huge news. Why isn't this more known? Is it moving forward? Has it been announced officially...
- dna_polymerase 6y agoMaybe because the repo is 2 hours old, provides no actual information other than some hand waving. Also the author asks for $2 Million and doesn't seem to be a Python Core Developer. Given the repos age I'm probably answering to Mark Shannon himself. Edit: This post wasn't supposed to be an hostile attack against anybody. Just found it odd to ask why an 2 hour old repo wasn't more known. I did not know that Mark Shannon is a core dev. Certainly did not pick that up from the repos contents.
- ebg13 6y agoI think Mark Shannon is a Python core dev now. But I agree with your other reasons.
- nemetroid 6y agoMark Shannon definitely is a core developer, though I agree that it's odd to ask "why isn't this more known?" about a two hours old repo.
- coldtea 6y agoHe is a core developer (and long time core contributor), and he doesn't ask for "$2 million dollars". In fact the post says: "The PSF seems to be the obvious organization to coordinate funding". And no, I'm not Mark, somebody send me the link. Conspiracy theory much?
- antoinealb 6y agoThe repository linked mentions that each stage should cost about $500k and there are 4 stages. You can argue that he is not asking for it, but at least he is valuating it at $2M.
- coldtea 6y agoDidn't say he isn't valuating it at $2M. It's all on the post. What he doesn't do is say he should be the one to handle those (or even get those).
- XorNot 6y ago200k per year would not be the worst salary to pay a developer for this work, and I'd say he's outlined about 5 to 10 years of work. Feels like you could negotiate a package of escalating compensation contingent on delivery milestones.
- pizza234 6y agoI personally think it's a bit premature to talk [edit: for the developers to talk] about it - they're attributing 2.25x to the addition of a JIT, which generally takes a long time to develop and tweak, adds a significant complexity, and makes the performance profile complex (ie. slow start, slow first requests, etc.). I like though, to see the JIT wave across interpreted languages (see Ruby). I'm also curious to see what's going to be the implementation.
- dna_polymerase 6y agoThe problem with this and all other efforts for speed in CPython is the interop with C extensions. For JITted Python look at PyPy, sure the JIT gives you some performance, but they broke all C extensions along the way.
- heavenlyblue 6y ago> they broke all C extensions along the way. It doesn't have to be this way though?
- ploxiln 6y agoMany C extensions work with PyPy now. I have a service that uses pylibmc and pycurl, and it works with PyPy. Those extensions didn't rewrite with cffi or anything like that, PyPy added a compatibility layer, it's impressive. But, you don't get the JIT performance if you use the C extensions a lot, it's better for PyPy performance if you find a pure-python alternative for extensions used in "hot paths".
- chrisseaton 6y agoThe TruffleRuby approach is to run the C extensions in an interpreter with a JIT. This way you can change the C extension interface without changing the code, and the optimisations apply to both at the same time. https://www.youtube.com/watch?v=YLtjkP9bD_U https://www.youtube.com/watch?v=YLtjkP9bD_U
- MaxBarraclough 6y ago
- yxhuvud 6y agoBecause it is on the idea stage without funding. My guess is that it will stay that way. That said, it is probably feasible to do some parts of it, similar to the 3x3 project Ruby has. I wouldn't be surprised if numpy glue is a lot more tractable for optimization than Rails, but the interaction with C libs would pose some serious issues.
- mtklein 6y agoThis reminds me of Unladen Swallow: https://www.python.org/dev/peps/pep-3146/ https://www.python.org/dev/peps/pep-3146/ (So not holding my breath.)
- throwaway_pdp09 6y agoQuote from link "With Unladen Swallow going the way of the Norwegian Blue..." I love all this kind of thing that you don't get in commercial software.
- nurettin 6y agoHappens all the time at googleplex.
- tom_mellior 6y agoThis looks less naive than Unladen Swallow's "let's throw LLVM at it and hope for the best".
- kzrdude 6y agoI think unladen swallow caused this improvement https://bugs.python.org/issue4753 https://bugs.python.org/issue4753 as an upstream effect which was a 15-20% interpreter speedup
- oefrha 6y agoThe only discussion I can find about this on python-dev/python-ideas: https://mail.python.org/archives/list/python-dev@python.org/thread/RDXLCH22T2EZDRCBM6ZYYIUTBWQVVVWH/ https://mail.python.org/archives/list/python-dev@python.org/... (Mark Shannon has been a core dev since 2018-05-15: https://devguide.python.org/developers/ https://devguide.python.org/developers/ though his involvement dates back to 2011 according to git log.)
- ebg13 6y agoI hope that discussion thread continues, because so far there's some really juicy sass happening in there. > > 1. I already have working code for the first stage. > I don't mean to be negative, or hostile, but this sounds like you are saying "I have a patch for Python that will make it 1.5 times faster, but you will never see it unless you pay me!" and > Where is your working code for the first stage?
- duckerude 6y agoHis response: > I believe that's how business works ;) > I have this thing, e.g an iPhone, if you want it you must pay me. > I think that speeding CPython 50% is worth a few hundred iPhones. I wonder if he considered publishing it under a restrictive license that doesn't allow real world use. Then people could scrutinize his claims but there would still be an incentive to pay up for him to relicense it.
- remote_phone 6y agoReleasing it before payment would be an insanely bad idea. 50k is about 3 months worth of a single FAANG developers base salary. It’s well worth it for someone to pay for the code.
- nlpx 6y agoThere are many other developers who have made huge contributions and have not asked for money. But in the new corporate Python world this might be the logical conclusion. The losers are the people who made Python what it is today, for free.
- rich_sasha 6y agoAtm this reads like a shopping list, _but_ if it goes ahead, and even if it doesn’t fully deliver, I think this could be the impulse that Python needs to remain relevant in scientific programming. Clearly it is the leader right now by a long way, but Julia is growing really fast.
- UncleOxidant 6y agoWas Julia also part of the impetus for the recent pep for adding macros?
- FridgeSeal 6y agoI remember asking this question here or on reddit when that popped up, and was rebuffed, but I do suspect Julia having powerful macros was a driver.
- spappal 6y agoThe author of that macro PEP [0] is the same Mark Shannon behind the speedup proposal under discussion. [0]: https://www.python.org/dev/peps/pep-0638/ https://www.python.org/dev/peps/pep-0638/
- cheph 6y agoI wonder why they don't just shift to JVM or even .net core or mono where I think a lot of this is already solved problems.
- coldtea 6y agoBecause then they have other problems...
- latenightcoding 6y agoI favorited this post to read the discussion later, but looking into that "plan", it's mostly a bunch of nothing. It's very hard to estimate how much speedup a JIT will get you on a dynamic language like python and x5 speedup seems unrealistic. There are other lower hanging fruits, like optimizing core data structures (e.g: the implementation of python dicts) .
- cuchoi 6y agoWhat makes you think that the implementation of dicts is a low hanging fruit? Only asking because there has been a lot of work to make them faster: https://www.youtube.com/watch?v=npw4s1QTmPg https://www.youtube.com/watch?v=npw4s1QTmPg
- jammycrisp 6y agoAgreed on optimizing core objects. I recently wrote a C base class (https://jcristharif.com/quickle/#structs-and-enums https://jcristharif.com/quickle/#structs-and-enums) for defining dataclass-like-types that's noticeably faster (~5-10x) to init/copy/serialize/compare than other options (dataclasses, pydantic, namedtuples...). For some applications I write this has a non-negligible performance impact, without requiring deep interpreter changes. Using the base class is nice - my application objects are still defined in normal python code, but all the heavy lifting is done in the c-extension. However, this speedup comes at the cost of being less dynamic. I'm not sure how much more optimized core python objects could be without sacrificing some of the dynamism some programs rely on. Python dicts are already pretty optimized as is.
- byko3y 6y ago>I recently wrote a C base class (https://jcristharif.com/quickle/#structs-and-enums https://jcristharif.com/quickle/#structs-and-enums) for defining dataclass-like-types that's noticeably faster (~5-10x) to init/copy/serialize/compare than other options (dataclasses, pydantic, namedtuples...) YouTube also encountered the same problem. Their solution sounds kinda like "never use pickle, because it's slow. Use custom serialization".
- metalliqaz 6y ago
- sciclaw 6y agoI hope this goes forward. Not just for the speed, but the energy benefits. One thing I love about C and golang is how fast they make hardware feel. They can do much more with less hardware. I love writing Python, but it does feel a bit heavy. If every machine using Python required half as much hardware/power that would be amazing.
- kzrdude 6y agoEvery python program starting 50% faster would be awesome (importing packages is slow)
- munificent 6y agoI've been observing something akin to Zawinski's Law ("Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.") but for programming languages: Every dynamically-typed programming language attempts to expand until it has static types and compiles to machine code. * Common Lisp has long had type annotations that can be use to generate specialized code. * JS started out as a simple bytecode VM, then got a JIT from V8 and others, then TypeScript came and gave it static types. * Python has had several JITs over the years—Unladen Swallow, PyPy, etc. It got static types with mypy (and others) and is now adding type annotations directly to the core language. * Ruby's 3x3 plan involves adding a type-specializing JIT. Sorbet adds static types and I believe Matz wants the core language to go in that direction. * Facebook created Hack to statically type PHP and PHP core added "scalar type dependencies". Facebook's HipHop VM brings a JIT to PHP. * LuaJIT brought a high performance JIT to Lua and there's been a number of projects that layer static type annotations onto the language. * Dart started with an optional type system and moved to a fully sound static type system. So what I see is a language that starts out simple and dynamically typed with a set of core libraries and idioms designed around dynamism. Then later people add static types on the front end to help people maintain larger programs. And they add type specializing JITs on the back end to generate faster code. But right in the middle you're still stuck with a mountain of existing code designed around the assumption that code and data don't need to be statically shaped. So even though you end up doing all the work (and adding all the complexity) to design a static type system and native code generator, you don't get the full benefits. The static type systems are almost always unsound in order to play nice with existing dynamic idioms, so the back end can't use the static types for optimization purposes. You end up with these fantastically complex type systems like TypeScript's and these incredibly complex JITs, but you still don't get the performance you get from a simple fully-statically typed language like Go or, hell, Pascal. I think the reasonable take-away is that if you ever intend your language to be used for large programs, just take the hit and start off with static types. Your future self will thank you.
- chubot 6y agoI think the reasonable take-away is that if you ever intend your language to be used for large programs, just take the hit and start off with static types. Your future self will thank you. I think this is missing something: the creators of those languages wanted languages that were fun/productive for writing small programs. That is, extremely flexible languages (and yes maybe they were unaware back in the 90's how this would limit their optimization potential) And all big programs were once small programs. Facebook is probably one of the easiest to see, since it was "just" a bunch of PHP scripts (although people tend to underestimate/dismiss it for that reason). Just like nobody ever says: "Well I estimate that in 10 years Facebook will consist of 10M lines of PHP with 5,000 programers working on it -- I should probably write it in another language". Nobody ever says: "Well I think my language is going to be used by millions of people and will have billions of lines of code". Well, there probably were people who thought that, but those were exactly the people who didn't make languages as useful as Python and JS :) ---- That said I think the phrase "irrational exuberance languages" referring to Python/JS is kinda funny, and in a way accurate ... https://blog.sigplan.org/2020/10/12/from-heavy-metal-to-irrational-exuberance/ https://blog.sigplan.org/2020/10/12/from-heavy-metal-to-irra... Although again I would say the unexpected part was not that they thought single core scaling would continue forever and make their languages fast, it's that those "slow" languages turned out to be the "best" ones for writing some of the most important systems of the last couple decades (not just commercial ones, but also Wikipedia, BitTorrent, etc.) ---- I'd make another analogy, to ISAs. If you talk to anyone who knows about CPU design, they'll say that x86 is shitty with a big pile of hacks. "It would be better" if someone designed it from the beginning with current applications in mind. But if anyone actually did that back in 1980, they wouldn't have been successful. And from my perspective, I mostly don't care, because the C compiler makes it all work for me (although I know the people who make it work care very much). So I guess the point is that technology adoption proceeds by evolution, and trying to plan 10 or 20 years ahead of time never works. ---- Also, there is a pretty hard tradeoff between static types and metaprogramming. Recent languages have come closer to reconciling these features (Zig, Nim, D), but dynamic languages chose reflection/metaprogramming, and that's a primary reason why they became successful. Ruby on Rails is a great example of that. It uses Ruby metaprogramming/reflection to a hilt, and lots of people who have no idea what that is love it, and they built tons of things with it (which now makes it an interesting optimization target)
- _zamorano_ 6y agoThere's already PEP 640, targeted for 3.10, that seems will break C-API compatibility. https://www.python.org/dev/peps/pep-0620/#the-c-api-blocks-cpython-evolutions https://www.python.org/dev/peps/pep-0620/#the-c-api-blocks-c... This seems like the first step towards such improvements, but I guess recompiling libraries will be mandatory, and already existing wheels won't be valid for Python 3.10 Isn't this more scary than the Python 3 migration?
- xapata 6y agoDude, I had to recompile all my extensions for every minor version of Python as recently as ... 2013? Wheels haven't been around for that long. I'd get a new laptop and curse, "Why didn't I think to write down all those Fortran compile flags I used for SciPy!"
- kzrdude 6y agoCouldn't Mark go for being employed in a sponsoring/hosting company for 6-12 months? It shouldn't be impossible to pitch this, I think.
- kmod 6y agoFrom my experience in this area: - There are companies which would be willing to pay large amounts of money (more than he's asking for) for the speedups he's promising - There are other similar efforts underway (such as ours) which have implemented some of his ideas and the gains are much smaller than he anticipates (for real code, microbenchmarks are another story)
- tln 6y agoWas your effort Pyston? https://github.com/pyston/pyston https://github.com/pyston/pyston
- mrich 6y agoThere needs to be a sensible way to measure progress, so a set of benchmarks needs to be chosen. Not sure all funders can agree on those. Also, I believe many large companies who are running code at truly large scale are already using low-level languages like C++ (Google).
- beagle3 6y agoRelated (though seems abandoned:) https://github.com/microsoft/Pyjion https://github.com/microsoft/Pyjion
- linsomniac 6y agoI feel the need. The need... for speed. See you in Iceland! https://wiki.python.org/moin/NeedForSpeed https://wiki.python.org/moin/NeedForSpeed Was just talking about that trip with my wife last night, in relation to COVID statistics. "Measuring things sucks! I'm still traumatized from spending a week trying to reliably measure speedups in Python."
- roca 6y agoThere isn't any discussion in the document of why this effort will succeed when previous efforts along very similar lines failed.
- mark-r 6y agoI have a method for turning integers into strings that is much faster than Python's built in method when used with very large integers. I wonder if there's a place for it in this project?
- bastawhiz 6y agoIf your method is performant, then you should be able to simply file a pull request to improve what's there. I don't see a reason you'd need to tie that contribution to any existing effort.
- mark-r 6y agoHaving never contributed to an open source project, particularly one as popular and complex as Python, I'm not sure of the amount of justification necessary. Would I need to write a PEP?
- whb07 6y agoAnyone comment the obvious low hanging fruit? Compile CPython in release mode, as most everyone uses the debug version. Doing this gives a rough 10% performance increase.
- macjohnmcc 6y agoI've compiled some code using Microsoft tools and saw some huge performance gains on code I didn't think would benefit.
- sgerenser 6y agoIf you’re talking about Debug vs Release mode in Visual Studio, there’s a huge performance hit with STL iterators in Debug mode. Like sometimes 10-20x slower for tight loops. This is obviously C++ not C, so it doesn’t apply to Python directly. But in general toolchains can have very unoptimized code paths in debug mode.
- kzrdude 6y agoThis seems like a joke? CPython uses a threaded interpreter loop, so surely they optimize the release build?
- RantyDave 6y agoYeah, I dunno. The more I work with Python (CPython) the more I'm starting to like that it can be used as a scripting layer over "really complicated stuff". IE if you want to store and mess about with some data, Python is fine. If you want to store and mess about with a lot of data, use pandas, numpy etc. etc.
- Waterluvian 6y agoHow do you estimate the performance improvement without actually implementing it first? Are any of those details published?
- est 6y agoAuthor's email contains an website, hotpy.org, which redirects to here https://sites.google.com/site/makingcpythonfast/ https://sites.google.com/site/makingcpythonfast/ Which leads to an now deleted repo https://bitbucket.org/markshannon/hotpy_2 https://bitbucket.org/markshannon/hotpy_2 Searching for HotPy on HN yields another now 404 URL: http://www.dcs.gla.ac.uk/~marks/ http://www.dcs.gla.ac.uk/~marks/
- jbritton 6y agoI don’t know anything about psyco, but is there some possibility of reviving that project. Is it just too much work?
- chenzhekl 6y agoI also suspect the viability at first, but later I saw Mark's prior work. Hope this can provide some background for the discussion: https://sites.google.com/site/makingcpythonfast/ https://sites.google.com/site/makingcpythonfast/
- quotemstr 6y agoAll the code speedups in the world won't help if the allocator remains slow and escape analysis doesn't yield a lot of stack allocation opportunities. A nice JIT (tracing or not) needs to be combined with a nice GC to generate the maximum benefit.
- antpls 6y agoWhile it's always an interesting topic, since I used Python for the first time 10 years ago, it has been tried and tried again to only realize the same thing again and again : 1) the Python language was never made for speed of execution, it was made for speed of programming. It is an awesome language for prototyping, teaching and scripting 2) If you want speed, it likely means you are doing math operations, for example ML. In that case, you'd better learn to use state-of-the-art math libraries, made of decades of hardware and mathematical expert knowledge you will never beat. It would serve you better to : - learn about existing state-of-the-art compiled librairies and tools in your problem space - learn to profile your Python code Python is a great orchestrator to glue libraries and external systems together. If you are reinventing everything in Python, you are not solving actual problems. It's matter of using the right tool for the right task. Python is a mean to achieve something greater, it's not an end by itself.