4 ms·
As a huge Python fan, I think now is the time for the leadership to start thinking about Python 4 (especially since the 2->3 migration headache seems to have mo
by MR4D 5y ago
As a huge Python fan, I think now is the time for the leadership to start thinking about Python 4 (especially since the 2->3 migration headache seems to have mostly gone away).
I say this because v4 would be a good time to look at putting some of these speed improvements into the mainline code - something that would be largely backward compatible, but make those breaking changes that give big speedups (such as putting limits on the C interfaces and other big building blocks).
My gut (and really, it's just a half-assed guess) tells me that Python could easily see a 2x speedup in exchange for some of those incompatibilities, and an even bigger gain depending on how deep the changes went. [0]
If that's anywhere near correct, then doing a v3/v4 in parallel (like v2/v3 was) would probably be the only way to go, so that people would know there is a migration path. [1] [2]
[0] - There are really only two ways to do this - keep breaking things and introducing incompatibilities along the way, or make a bigger break for the sake of performance. Of course, doing nothing is also an answer, but probably one that we wouldn't really be happy with. There is also a variant of versioning where we do 4/5/6/... in relatively rapid succession, such as 2-3 year intervals, and make a fewer big breaks for each release, and that might even work better.
[1] - I suffered through the v2/v3 issues also, (as an end user more than a dev though), and it wasn't fun. But I'd do it all again if it meant a big jump in speed.
[2] - I know the perl team mucked this up (hindsight being 20/20), so having a working V4 on first release (not counting alphas & betas) would be key. Scoping that out well beforehand would probably be needed, but I think the team is broad and deep enough to do that.
- troutwine 5y agoGiven how troublesome the 2->3 migration has been for large companies I wonder if a breaking 3->4 wouldn’t kill the language. I know of more than one big shop that still has a lot of Python 2 in house but has disallowed the creation of new Python code bases in favor of other languages.
- deadmutex 5y agoWhat is the definition of a "big shop" here?
- troutwine 5y ago50k person years of work represented in the code bases and up, as an approximation. Big enough that to do the work you'll have to coordinate across multiple departments with different incentives for multiple years to get the transition made.
- MR4D 5y agoI've thought this over, and my conclusion seems to be that people will take a reasonable amount of stress if they get a big performance improvement. I absolutely agree that for some people, it will be too much, and frankly, I don't think that problem is solvable.
- troutwine 5y agoWell, it's solvable in the sense that breaking language compatibility is a choice that doesn't have to be made, I guess. The 2->3 transition was managed as effectively as such a big thing could be but it's for left the industry littered with a bunch of weird self-maintained python2 forks, compatibility layers for the C side of python. A 2x theoretical performance improvement doesn't pay down the engineering costs to make a switch if you recoup it in lower hardware costs later.
- hitpointdrew 5y ago>hat still has a lot of Python 2 in house but has disallowed the creation of new Python code bases in favor of other languages. That seems silly. It's not like other languages don't have their own major version migration pains.
- reom_tobit 5y agoI wonder whether it would be better to just go with the nuclear option and kill the C extension model as it currently stands. It would take probably decades for libraries to catch up, and maybe kill the language, but if it survived, Python would be so much more attractive for many kinds of projects. Conversely, Python has a lot of niches, and works well for them. Incremental improvements are probably fine, even with the dreaded GIL.
- nyanpasu64 5y agoWhat would be gained by killing the C extension model? Making it easier for PyPy/GraalPython or the more recent Pyston and Cinder to outperform CPython? HPy is in the process of creating a new C extension API that doesn't expose CPython internals and is easier for alternative Python implementations to support with high performance.
- maccam94 5y agoI'd say 2x probably isn't worth causing major ecosystem pain. Maybe if it opened up potential for 10-100x speedup for some fairly common workloads or improved developer velocity.
- testesttest 5y agoI agree. Servers are cheap compared to people. If we had to endure another large python migration, we would just migrate to a different language with better perf.