3 ms·
Officially, as in I can't point you to this in writing off the top of my head but I've heard Guido say it numerous times, part of the design goal of CPython is
by kstrauser 6y ago
Officially, as in I can't point you to this in writing off the top of my head but I've heard Guido say it numerous times, part of the design goal of CPython is to be as understandable and approachable as possible. That code you linked doesn't include stuff like compiling opcodes directly to assembly and then executing them natively, but it is very straightforward. I mean, I'm not a C guru by any stretch but I can look at that and tell immediately how I could go about adding my own opcodes to the VM.
- throwaway894345 6y agoPerhaps, but it's unfortunate that so many of Python's design objectives are (in practice if not in theory) so limiting with respect to performance, tooling, package management, maintainability, and other important dimensions of professional software development. Of course, it doesn't mean there aren't companies who can't be successful in spite of these limitations--if you make enough money you can afford to throw money at these problems and make them better (you don't need efficient compute or parallelism if you can afford to pay a team to set up and operate a Spark cluster for you not to mention the cost of the additional VMs and the additional effort to write spark instead of a native language).
- hn_acc_2 6y agoNot sure about the first part...How is Python's approachable design "so limiting" to all those dimensions? Nobody writes performance-critical code in pure python. Not sure how "tooling" is bad, what would you say is limited there? Package management, again, what package management problems are unique to Python? Many people say this but it seems the problems they bring up are not unique to pip or the python ecosystem, same problems are found with rubygems, npm, Maven, etc... Maintainability is a responsibility of developers and not a programming language, and unmaintainable code can easily be written with any language. However I'd argue Python should score positive points for maintainability; one of the languages I feel most comfortable picking up old code from someone else and groking it easily.
- throwaway894345 6y ago> How is Python's approachable design "so limiting" to all those dimensions? One of Python's "design goals" is to integrate well with C by exposing every detail of the Python interpreter to C extensions. Since Python's performance is so abysmal, the ecosystem has come to lean heavily on C-extensions, and because the ecosystem leans so heavily on C-extensions, very few changes can be made to the CPython interpreter without breaking compatibility with the ecosystem--if we can't change the interpreter, we can't optimize it, and we're damned to a world in which Python is slow. Pypy is doing yeoman's work by building a new JITed interpreter that makes pure-Python code a lot faster, but it still has a lot of compatibility problems with important C packages. For instance, you can't talk to a Postgres database without using an obscure package that hasn't seen a commit in years. > Nobody writes performance-critical code in pure python. Not successfully, no. But if you buy the marketing, you'll be led to believe that Python has an answer to every performance problem. "Go ahead and start your project in Python. Don't worry about performance--if your program is too slow, you can just $X" where "X" is one of "rewrite the slow bits in C" or "use pandas" or "use multiprocessing". No one tells you that those options really fall over for a huge swath of real-world workloads, for the same reason: in many/most cases, the cost of de/serializing Python objects is greater than the savings from C or parallelism. It's only economical for those precious few cases where you can do a lot of consecutive work outside of Python. > Not sure how "tooling" is bad, what would you say is limited there? Precious little static analysis is available for Python, documentation generation options are generally bad (partly because they can't just generate the type information per the previous static analysis point, but also because they make a whole host of bad decisions, like putting everything on the same page and making you scroll around to figure out which class's __init__ method you're looking at presently) and you're still on the hook for operating the CI tools that generate and publish the documentation packages, tools tend to be written in Python and thus are really slow (e.g., formatters, package managers, etc), no static analysis means no dead code elimination and thus an enormous installed footprint well into the hundreds of megabytes (this problem is exacerbated by the weight of OOP in the Python ecosystem, which means everyone who wants a Book data structure depends on the whole universe of things that people do with Books), static distribution is still a joke--you end up bundling 250mb zip files and you still need to have the right version of Python and the right .so/.dlls installed on the target system, etc. > Package management, again, what package management problems are unique to Python? In order to figure out what a Python package's dependency tree looks like, you have to download the whole thing. This makes it difficult to have performant package managers (or rather those that are performant are unsafe because they splat things into the python environment and punt on making sure they don't have multiple versions of some transitive dependency). Python is also held hostage by an ecosystem of C-extensions, so its packages have to support the whole universe of terrible C package management decisions. Also, there is still no production-ready Python package manager that supports reproducible builds (i.e., respects lockfiles). I don't know about Ruby, but NPM, Go, and Rust don't have these issues and I'm pretty sure Java, C#, and Ruby don't either. > Maintainability is a responsibility of developers and not a programming language, and unmaintainable code can easily be written with any language It's a lot easier in a language without any rails to guide developers toward good development practices (by making it disproportionately harder to write hacky code), by which I mostly mean a static type system. Mypy is gaining traction, but it's still not in the same ballpark as other languages' type systems and it's moving at a snail's pace (no doubt other languages benefit from investment or else static typing was built into the original design, but those excuses don't make my team's code more maintainable). Not only is it rails to guide them on the right path, but it's also things like "type documentation is always correct" and "refactoring is easy so people actually do it".
- kstrauser 6y agoI get what you’re saying, but in practice I’ve been using Python for a while, since 1.x, and I haven’t run into many of those potential issues. Some people have, obviously, and I wouldn’t discount them. I think the differentiator is that while Python isn’t great when you care about every scrap of performance, there are a huge number of problems where that isn’t a requirement. For instance, my employer’s web stack is built around Python and profilers show that we spend less than 5% of a request’s time running Python code. The other 95% is waiting for database queries and stuff like that. For us, Python is an excellent fit because it’s really good at the other parts of API server development and we’re nowhere near CPU bound. And turns out, a lot of websites have realized that they’re in the same boat. I wouldn’t recommend pure Python (that is, without Numpy or Scipy) for machine learning, but it’s great in its own niches which are actually pretty enormous.
- throwaway894345 6y ago> I think the differentiator is that while Python isn’t great when you care about every scrap of performance This is an understatement. Python is the slowest mainstream language. JS is about an order of magnitude faster. Go, C#, Java, etc are about 2-3 orders of magnitude faster. C, C++, and Rust are nearly 4 orders of magnitude faster. If you might ever care about performance at all, Python is limiting. No doubt that if all you're ever doing is a little glue code that dispatches to a database (or to ML models or whatever), then Python is fine. The problems come about when you try to do more interesting things, especially involving a complex Python object model that doesn't map neatly to Postgres or Pandas. I've never worked on a Python project where this wasn't the case, and where the organization wasn't doing some desperate optimization (e.g., standing up a Spark cluster) to stave off a ridiculously gracious timeout limit (e.g., 60s for browser timeouts). Meanwhile, had the same server been written in Go, the naive implementation would have subsecond response times and a little bit of trivial optimization (the kinds of optimizations that aren't available in Python for a myriad of reasons) could get it down to double digit milliseconds. Since we rarely know all of the types of bottlenecks a project will encounter in its years of service, Python has become something of a last resort for me (especially given all of the great options that exist nowadays). And none of this is to say anything about the tooling, distribution, or package management problems that plague Python either.
- Jasper_ 6y agoBut it's not like every tiny optimization will turn the runtime into unreadable soup. But that's been the defense every time someone proposes adding some extremely basic optimizations [0]; these things aren't difficult and any VM engineer will understand them. And even if not, it's not that that hard to learn about basic method caches. Write a few blog posts, or a giant block comment. Stuff's not arcane magic, it's the foundation of any PL implementation these days. There are also a lot of programming language who don't bother touching Python because the team doesn't seem to care about runtime performance as their responsibility. There's a lot of excellent talent out there, and they're mostly being denied and shot down. [0] https://lwn.net/Articles/754163/ https://lwn.net/Articles/754163/
- kstrauser 6y agoI do agree with you on that. I think there's a happy path where you keep the high-level architecture clean and approachable, and move the tricky bits out to the edge where only a few people will have to know and care about them. That seems to work out pretty well for PostgreSQL, which on the surface is purely black magic but by all accounts is quite nice under the hood.