4 ms·
It’s interesting to me that Python depends so much on native code to have even reasonable performance, but because so much of the ecosystem is native code and b
by throwaway894345 6y ago
It’s interesting to me that Python depends so much on native code to have even reasonable performance, but because so much of the ecosystem is native code and because the native code interface is “CPython shaped”, it becomes incredibly difficult to improve CPython’s performance because it would very likely change the shape of CPython. Of course we could have the best of both worlds—a fast Python that allowed you to use native extensions—but the native extension interface would have to be minimized to something that would give Python implementations some ability to look different than CPython today. And when you have a performant Python implementation, much less needs to be implemented natively, which makes things like package management and cross platform development much easier (specifically you don’t have to worry about cross-compiling and testing on your target becomes considerably less important because Python is portable while native extensions likely are not). The latter becomes a lot more pertinent if you’re on x86 targeting ARM or if you’re on a new ARM MacBook targeting x86 or if you’re targeting WASM from either. These cases aren’t common today, but ARM and WASM seem to becoming increasingly likely in the future.
- mumblemumble 6y agoI think that what's even more interesting to me is that people perceive this as such a preoccupying problem. When I was a young little programmer, the dream was to have a base layer of highly performant native code for handling the heavy lifting, tightly coupled to a very high level scripting language that wasn't really even trying to be performant. The thought being that the scripting language should be more focused on flexibility than performance, because, for the bits of the software that it handled, developer productivity was more important than raw performance. Fast forward a year or two, and we are living the dream. It's Python, and, while I could nitpick implementation details and language features all day, the big picture is that it's pretty awesome. So awesome that, for the machine learning and data engineering work that I do, the most performant and most productive language I have to work in is Python. (Also R, which is a pretty similar technical story, but I realize that R lost the popularity contest, and I understand why.) I guess it starts to feel to me like we're making the perfect the enemy of the good, here. We've got a well-established, productive, flexible, and easy-to-learn language where top-tier performance can generally be implemented as a library solution. Despite its many imperfections, that's pretty darned good. In my career to date, I haven't seen much cause to believe that I could realistically expect much better than that. I think that, when I do need better performance, I'd rather write a library than rewrite the language.
- throwaway894345 6y agoSome people have performance requirements to meet, and we’re pretty locked into Python (because some of our engineers foolishly bought into the promise that we could just rewrite the slow parts in C/Pandas/multiprocessing/etc). If this is “living the dream” then Go and other languages will blow your mind. You can have iteration that is at least as fast (especially when collaborating with other engineers) and performance that is 2-3 orders of magnitude better than Python. Not to mention a better packaging and deployment story. The sad thing is Python could be much better, but it has to back out of some bad design decisions first.
- ris 6y ago> Not to mention a better packaging and deployment story. Go's "packaging and deployment" is enough to rule it out for new projects for me.
- ska 6y agoThis was known at the dawn of python and before, though. For example lisps in various flavors were doing this high-and-low very well decades earlier; arguably nothing has done it radically better since. So it's worth asking why that never took over the world as a counter to why python exploded. It's tempting as a technology person to wish a "better python" had been the one to gain that sort of market impact. However, we have to at least ask the question if some of its technical flaws actually contributed to its success....
- throwaway894345 6y ago> So it's worth asking why that never took over the world as a counter to why python exploded. JavaScript would like a word.
- cheez 6y agodo you mind sharing exactly what problems you've had with Python?
- 6y ago