6 ms·

It's bizarre. I don't think it's an exaggeration that it's the 10th project I've heard about to speed up Python. Seriously, use a faster language. If you need

It's bizarre. I don't think it's an exaggeration that it's the 10th project I've heard about to speed up Python. Seriously, use a faster language. If you need a performant fork of Python, you're using the wrong tool for the job.

actionfromafar 4y ago
There isn't much inherent in the language design which dictates it must be very slow, though. It's mostly a result of the canonical implementation.
- yegle 4y ago
  To have different implementations we first need a "Python language specification" which doesn't exist. All other implementations have to be compatible with CPython to increase their chance of getting wider adoptions.
- pjmlp 4y ago
  Indeed see Smalltalk, SELF, Common Lisp.
- bakal 4y ago
  Then how come that there have been so many forks focused on speed yet none has managed to get good enough?
  - anakaine 4y ago
    Good enough in what way, specifically? Python is exceptionally popular not because of speed, but because of its wide compatibility, extensive list of maintained libraries, good learning paths, and most likely because its fast enough for most things. Every iteration of Python for some time now has added a not insignificant speed boost. It's not as fast as C, and nor will it be given how it works, but it certainly is fast enough for a great many situations and tasks. All of the above doesn't make it the right tool for every job. Sometimes you need to hit a nail, and you don't have a hammer but you do have a lump of hardwood. And in that situation, you make an assessment if in the long term you need to hit more nails regularly so you invest in a hammer, or just occasionally and you're ok with the lump of wood. It's subjective. So is it good enough? Good enough for what?
    - thehappypm 4y ago
      The speed of the programming language, in many applications, is a lot less relevant than the overall architecture. For example if you’re running a Web server and you’re serving a request through some hyper optimized C program, that is “fast”. But using a CDN in front of a slow Python server is almost always going to be faster for end users. Point is, the application is often just one part of a system. Python might let you develop quickly and avoid footguns, and having one slow link in the system doesn’t mean the system as a whole is gonna suck
- jakobnissen 4y ago
  I disagree. Python is inherently slow: * It uses an interpreter, which is slower than compiled code * All objects carry a dict to contain arbitrary attributes - this creates huge overhead * Even when the dict is suppressed with __slots__, all containers (all classes, even!) are untyped, so extracting any field causes dynamic dispatch * Objects cannot be stored inline in containers * Integers are variable-sized, and thus slower. Of course, you might counter that one could simply make a version of Python which is compiled, where all classes have a well-defined typed layout, where some classes can be immutable for performance, containers are typed, integers are machine integers, etc. But then I'd argue you're talking about a completely different language (Julia fits this description perfectly)
  - itsoktocry 4y ago
    And despite all of this, Python has generated billions (trillions?) of value, and continues to do so. Perhaps it's you who is focussed on the wrong things? Maybe some of these issues can be improved over time? Maybe none of those things actually matter in the real world, outside of niche use cases?
    - jakobnissen 4y ago
      Are you arguing about whether Python is a good language or whether it's a slow language? The former is a matter of taste, the latter is objectively true. And there are good reasons to believe it will remain true. It's created billions of dollars in value, sure. So had JavaScript and PHP. Doesn't mean they're good languages. Mediocrity gets you far, if you're at the right place at the right time.
  - lozenge 4y ago
    Sounds like Cython. PyPy did also fix some of those issues - specifically, lists of small integers are optimised, with a deopt path for when they overflow.
    - jakobnissen 4y ago
      All my points above are about the language semantics of Python, not the implementation. Some of them may be optimized some of the time with a clever enough JIT - but only a few of these points. That's why PyPy is not as fast as C.
- marcyb5st 4y ago
  I disagree as well. I was working at YouTube during the rewrite from python to c++ and a lot of people at Google looked at the problem to avoid a full rewrite of a major product. We still ended up doing it regardless of the pain of using Swig to wrap C++ code. Not missing that at all.
pjmlp 4y ago
When we created our own version of AOLServer, also with Tcl, I quickly learned that interpreted language + continuously rewriting code into C, wasn't the way for doing distributed systems where performance matters. Indeed this experience eventually made us jump into .NET as we had early access to it as Microsoft partners, and a couple of years later some of the people on this team went off and founded OutSystems. 20 years later, apparently this lesson of using scripting languages without dynamic compilers seems to have to be relearnt the hard way.
asutekku 4y ago
People start using python because of the syntax (and the ecosystem built around it ofc), changing the language doesn't solve the problem.
- Jensson 4y ago
  Dynamic languages aren't just about the syntax, they are also about copious amounts of implicit casts and slow but flexible data structures to make working with them possible. Python has a very good set of data structures and implicit casts, that is the main reason it is popular. Compare to jagascript for example... Anyway, making python fast while keeping all those casts and data structures is very hard, Javascripts casts and data structures are bad for the programmer but very good for the interpreter so is much easier to optimize which is why it runs orders of magnitude faster.
pizza234 4y ago
The same is happening at other major tech companies, e.g. Shopify. They've very likely made their calculations, and assessed that paying engineers for speeding up the language, was overall preferrable to rewrite part of their (very large) systems and potentially extract them into separate services. It's unlikely that they just cargo culted. Github/Gitlab, interestingly, did not take this road, and extracted slower part into Go services. Stripe had the (IMO) most interesting idea - write a (Ruby AOT) compiler for the slow paths. It'd be very interesting to see the precise/concrete considerations behind these choices, since generic considerations ("we love Ruby" :)) aren't very informative.
- jakobnissen 4y ago
  To me it looks like lock-in. They chose a language good for prototyping and quick iteration, and then their codebase gets stuck with a permanent performance problem. You see the same problem in Python with regards to correctness - it's hard to refactor Python or change a large codebase and have it keep working correctly, so huge Python projects tends to ossify. It may be a rational solution only in the short-term, but still an objectively bad solution overall.
  - pizza234 4y ago
    > it's hard to refactor Python or change a large codebase and have it keep working correctly, so huge Python projects tends to ossify. Although I don't work in Python, in general, projects written in dynamic languages can be refactored if they've been developed in a disciplined fashion, that is, with substantial test coverage (I guess that in modern projects, with type checking, the need is significantly reduced). This is not unrealistic (I'm currently worked on a large-but-not-huge project, and it's perfectly possible to refactor), although of course, whether dynamic language with lots of testing is still more efficient (in terms of engineering resources) than static language with less testing, it's another story.
    - guenthert 4y ago
      I was working on a code base where testing was inconvenient (re-striping a large RAID array took hours) and I can think of scenarios where testing is impossible (or at least very costly). Test driven development isn't always the right approach.
chpmrc 4y ago
Why? The language is great, it's the runtime that needs some work. We have bun because someone thought kind of the same thing about Javascript and decided to improve the runtime rather than to learn Rust.
- staticassertion 4y ago
  I'd be interested in hearing from others, but I wonder what differences there are for optimizing JS vs Python. A few things come to mind: 1. Python has a ton of C code in its libraries/stdlib, JS did not have that for the vast majority of its usage (browser). Optimizing across the FFI boundary seems unlikely to be a reasonable goal and may complicate optimization opportunity. Let's say you want to do some sort of optimization pass to make sure that `str.append` is not getting overridden - how are you going to handle native code? 2. Python is obscenely dynamic. Runtime generation of classes is not uncommon at all - for example, `@dataclass(slots=True)` creates an entirely new class. You can overwrite methods, functions, there's heavy use of globals in libraries, no such thing as private implementations, hooks into everything, even importing code has side effects so you can't reorder or lazily import automatically, even reordering imports is unsafe in Python. It seems really hard to optimize Python at all, and I wonder if we could ever reasonably expect to see a "V8-like" runtime, even assuming that there's funding, which I also doubt. Unless you're willing to break the language I just don't see a Python runtime overcoming these issues. Or introduce something like 'strict' where these things aren't possible.
  - pjmlp 4y ago
    It isn't more dynamic than either Smalltalk or SELF, whose JIT research ended up on the JVM and JavaScript. A a become: b message completely changes all instances of a to b, across the whole image, including code currently being executed.
    - lioeters 4y ago
      > Smalltalk or SELF, whose JIT research ended up on the JVM and JavaScript That makes me curious - would you have a link or hint where I could learn more on this topic?
lozenge 4y ago
Part of the reason is CPython left performance gains on the table for so long, in favour of implementation simplicity. It wasn't that difficult to improve on its performance. That policy seems to have changed now, though.
- guenthert 4y ago
  But not just for simplicity of implementation. There is no TCO for example, because "it makes debugging more difficult". Fast, easy, safe. Pick two.
- BiteCode_dev 4y ago
  Well, in 2008 the PSF had a budget of $500k for the entire world. To compare with JS, this is the order of magnitude of the salary of one single person working on google chrome. The community didn't have a lot of resources for a long time, and so had to prioritize: it chose to focus on python strenghts, and not compensate for weaknesses. It couldn't do both. Keeping the implementation simple was also a way to keep the project manageable on the cheap (and by cheap in mean in work hours, since the first 15 years, most core devs were just doing it for free). Today, a lot of companies with a lot of money have much to gain to invest in python, and so we can finally have our cake and eat it too.
BiteCode_dev 4y ago
It makes sense to me. Nowadays, all the GAFAM has solved most problems python is good at: glueing a lot of things, experimenting with ideas, bootstrapping new product, hiring and training a lot of new people fast. They now are in the optimization phase. They first picked the low hanging fruits, but after 10 years of doing that they need to fine tune. For them, gaining 20% of speed can add up to millions of dollar a day, so it's now the new priority. Just like with JS in the 2000', today they have to decide if it's worth rewritting python code or just improve the platform. For some projects, they made the calculations and it was just cheaper to improve the plateform. We got V8. Now Python is getting the new focus.
draaglom 4y ago
It's not that bizarre. It's path dependence. They have an existing codebase with hundreds (thousands? tens of thousands?) of person-years of effort contributed in python already, so a complete rewrite would be too expensive to consider.
- jakobnissen 4y ago
  The question is why they chose Python for apparently performance sensitive problems in the first place. The fact that they're now bumping against a performance wall and can't get around it by rewriting is a huge drawback when choosing Python as the language in the first place. Awareness that organisations often end up in these problems should be considered when choosing the language for a green field project.
  - draaglom 4y ago
    The decision likely got made before performance was a major constraint. As I understand it, Instagram is a python codebase and (guessing) may be the primary motivator here. Probably the first, say, 100 person-years of effort poured into the Instagram codebase were done at a time when product development velocity, not performance, was the main priority.
  - xiphias2 4y ago
    Instagram wasn't performance sensitive at all. It was iteration sensitive: if the team hadn't done it, some other group of people would have sold a company for billions of dollars to Facebook.
ehutch79 4y ago
As someone who uses python in their day job, I agree with this. When you spend more time io bound, it’s not an issue. If your python code is choking on cpu time, a lower level language is probably more appropriate.
- gpderetta 4y ago
  As someone who uses python in their day job, "IO bound" is a lie that is perpetuated to cover for the failing of the language. IO is very fast these days and python has the capacity of choking the cpu even on genuinely io bound problems . For example, do you think that python is an appropriate language to write pylint?
  - dekhn 4y ago
    By definition, an IO bound problem does not "choke the CPU", because at that point, it would be CPU-bound.
    - gpderetta 4y ago
      Sure, I'm claiming that many problems that would be easily IO bound on other languages, are in fact CPU bound in python.
      - ehutch79 4y ago
        Then use a different language? I thought I said that?
  - ehutch79 4y ago
    Network DB access is what I'm thinking about, not reading less than a K from on disk file. Also, any external API is an example. To answer your question; "it depends". Is pylints speed an issue? If not, then it's fine, if it is, the sure write it in go or rust or whatever. Javascript has a great example of this in esbuild.
itsoktocry 4y ago
>Seriously, use a faster language Some of the best engineers at the world's leading tech companies need to "pick another language", obviously.

actionfromafar 4y ago

There isn't much inherent in the language design which dictates it must be very slow, though. It's mostly a result of the canonical implementation.

yegle 4y ago
To have different implementations we first need a "Python language specification" which doesn't exist. All other implementations have to be compatible with CPython to increase their chance of getting wider adoptions.
pjmlp 4y ago
Indeed see Smalltalk, SELF, Common Lisp.
bakal 4y ago
Then how come that there have been so many forks focused on speed yet none has managed to get good enough?
- anakaine 4y ago
  Good enough in what way, specifically? Python is exceptionally popular not because of speed, but because of its wide compatibility, extensive list of maintained libraries, good learning paths, and most likely because its fast enough for most things. Every iteration of Python for some time now has added a not insignificant speed boost. It's not as fast as C, and nor will it be given how it works, but it certainly is fast enough for a great many situations and tasks. All of the above doesn't make it the right tool for every job. Sometimes you need to hit a nail, and you don't have a hammer but you do have a lump of hardwood. And in that situation, you make an assessment if in the long term you need to hit more nails regularly so you invest in a hammer, or just occasionally and you're ok with the lump of wood. It's subjective. So is it good enough? Good enough for what?
  - thehappypm 4y ago
    The speed of the programming language, in many applications, is a lot less relevant than the overall architecture. For example if you’re running a Web server and you’re serving a request through some hyper optimized C program, that is “fast”. But using a CDN in front of a slow Python server is almost always going to be faster for end users. Point is, the application is often just one part of a system. Python might let you develop quickly and avoid footguns, and having one slow link in the system doesn’t mean the system as a whole is gonna suck
jakobnissen 4y ago
I disagree. Python is inherently slow: * It uses an interpreter, which is slower than compiled code * All objects carry a dict to contain arbitrary attributes - this creates huge overhead * Even when the dict is suppressed with __slots__, all containers (all classes, even!) are untyped, so extracting any field causes dynamic dispatch * Objects cannot be stored inline in containers * Integers are variable-sized, and thus slower. Of course, you might counter that one could simply make a version of Python which is compiled, where all classes have a well-defined typed layout, where some classes can be immutable for performance, containers are typed, integers are machine integers, etc. But then I'd argue you're talking about a completely different language (Julia fits this description perfectly)
- itsoktocry 4y ago
  And despite all of this, Python has generated billions (trillions?) of value, and continues to do so. Perhaps it's you who is focussed on the wrong things? Maybe some of these issues can be improved over time? Maybe none of those things actually matter in the real world, outside of niche use cases?
  - jakobnissen 4y ago
    Are you arguing about whether Python is a good language or whether it's a slow language? The former is a matter of taste, the latter is objectively true. And there are good reasons to believe it will remain true. It's created billions of dollars in value, sure. So had JavaScript and PHP. Doesn't mean they're good languages. Mediocrity gets you far, if you're at the right place at the right time.
- lozenge 4y ago
  Sounds like Cython. PyPy did also fix some of those issues - specifically, lists of small integers are optimised, with a deopt path for when they overflow.
  - jakobnissen 4y ago
    All my points above are about the language semantics of Python, not the implementation. Some of them may be optimized some of the time with a clever enough JIT - but only a few of these points. That's why PyPy is not as fast as C.
marcyb5st 4y ago
I disagree as well. I was working at YouTube during the rewrite from python to c++ and a lot of people at Google looked at the problem to avoid a full rewrite of a major product. We still ended up doing it regardless of the pain of using Swig to wrap C++ code. Not missing that at all.

pjmlp 4y ago

When we created our own version of AOLServer, also with Tcl, I quickly learned that interpreted language + continuously rewriting code into C, wasn't the way for doing distributed systems where performance matters. Indeed this experience eventually made us jump into .NET as we had early access to it as Microsoft partners, and a couple of years later some of the people on this team went off and founded OutSystems. 20 years later, apparently this lesson of using scripting languages without dynamic compilers seems to have to be relearnt the hard way.

asutekku 4y ago

People start using python because of the syntax (and the ecosystem built around it ofc), changing the language doesn't solve the problem.

Jensson 4y ago
Dynamic languages aren't just about the syntax, they are also about copious amounts of implicit casts and slow but flexible data structures to make working with them possible. Python has a very good set of data structures and implicit casts, that is the main reason it is popular. Compare to jagascript for example... Anyway, making python fast while keeping all those casts and data structures is very hard, Javascripts casts and data structures are bad for the programmer but very good for the interpreter so is much easier to optimize which is why it runs orders of magnitude faster.

pizza234 4y ago

The same is happening at other major tech companies, e.g. Shopify. They've very likely made their calculations, and assessed that paying engineers for speeding up the language, was overall preferrable to rewrite part of their (very large) systems and potentially extract them into separate services. It's unlikely that they just cargo culted. Github/Gitlab, interestingly, did not take this road, and extracted slower part into Go services. Stripe had the (IMO) most interesting idea - write a (Ruby AOT) compiler for the slow paths. It'd be very interesting to see the precise/concrete considerations behind these choices, since generic considerations ("we love Ruby" :)) aren't very informative.

jakobnissen 4y ago
To me it looks like lock-in. They chose a language good for prototyping and quick iteration, and then their codebase gets stuck with a permanent performance problem. You see the same problem in Python with regards to correctness - it's hard to refactor Python or change a large codebase and have it keep working correctly, so huge Python projects tends to ossify. It may be a rational solution only in the short-term, but still an objectively bad solution overall.
- pizza234 4y ago
  > it's hard to refactor Python or change a large codebase and have it keep working correctly, so huge Python projects tends to ossify. Although I don't work in Python, in general, projects written in dynamic languages can be refactored if they've been developed in a disciplined fashion, that is, with substantial test coverage (I guess that in modern projects, with type checking, the need is significantly reduced). This is not unrealistic (I'm currently worked on a large-but-not-huge project, and it's perfectly possible to refactor), although of course, whether dynamic language with lots of testing is still more efficient (in terms of engineering resources) than static language with less testing, it's another story.
  - guenthert 4y ago
    I was working on a code base where testing was inconvenient (re-striping a large RAID array took hours) and I can think of scenarios where testing is impossible (or at least very costly). Test driven development isn't always the right approach.

chpmrc 4y ago

Why? The language is great, it's the runtime that needs some work. We have bun because someone thought kind of the same thing about Javascript and decided to improve the runtime rather than to learn Rust.

staticassertion 4y ago
I'd be interested in hearing from others, but I wonder what differences there are for optimizing JS vs Python. A few things come to mind: 1. Python has a ton of C code in its libraries/stdlib, JS did not have that for the vast majority of its usage (browser). Optimizing across the FFI boundary seems unlikely to be a reasonable goal and may complicate optimization opportunity. Let's say you want to do some sort of optimization pass to make sure that `str.append` is not getting overridden - how are you going to handle native code? 2. Python is obscenely dynamic. Runtime generation of classes is not uncommon at all - for example, `@dataclass(slots=True)` creates an entirely new class. You can overwrite methods, functions, there's heavy use of globals in libraries, no such thing as private implementations, hooks into everything, even importing code has side effects so you can't reorder or lazily import automatically, even reordering imports is unsafe in Python. It seems really hard to optimize Python at all, and I wonder if we could ever reasonably expect to see a "V8-like" runtime, even assuming that there's funding, which I also doubt. Unless you're willing to break the language I just don't see a Python runtime overcoming these issues. Or introduce something like 'strict' where these things aren't possible.
- pjmlp 4y ago
  It isn't more dynamic than either Smalltalk or SELF, whose JIT research ended up on the JVM and JavaScript. A a become: b message completely changes all instances of a to b, across the whole image, including code currently being executed.
  - lioeters 4y ago
    > Smalltalk or SELF, whose JIT research ended up on the JVM and JavaScript That makes me curious - would you have a link or hint where I could learn more on this topic?

lozenge 4y ago

Part of the reason is CPython left performance gains on the table for so long, in favour of implementation simplicity. It wasn't that difficult to improve on its performance. That policy seems to have changed now, though.

guenthert 4y ago
But not just for simplicity of implementation. There is no TCO for example, because "it makes debugging more difficult". Fast, easy, safe. Pick two.
BiteCode_dev 4y ago
Well, in 2008 the PSF had a budget of $500k for the entire world. To compare with JS, this is the order of magnitude of the salary of one single person working on google chrome. The community didn't have a lot of resources for a long time, and so had to prioritize: it chose to focus on python strenghts, and not compensate for weaknesses. It couldn't do both. Keeping the implementation simple was also a way to keep the project manageable on the cheap (and by cheap in mean in work hours, since the first 15 years, most core devs were just doing it for free). Today, a lot of companies with a lot of money have much to gain to invest in python, and so we can finally have our cake and eat it too.

BiteCode_dev 4y ago

It makes sense to me. Nowadays, all the GAFAM has solved most problems python is good at: glueing a lot of things, experimenting with ideas, bootstrapping new product, hiring and training a lot of new people fast. They now are in the optimization phase. They first picked the low hanging fruits, but after 10 years of doing that they need to fine tune. For them, gaining 20% of speed can add up to millions of dollar a day, so it's now the new priority. Just like with JS in the 2000', today they have to decide if it's worth rewritting python code or just improve the platform. For some projects, they made the calculations and it was just cheaper to improve the plateform. We got V8. Now Python is getting the new focus.

draaglom 4y ago

It's not that bizarre. It's path dependence. They have an existing codebase with hundreds (thousands? tens of thousands?) of person-years of effort contributed in python already, so a complete rewrite would be too expensive to consider.

jakobnissen 4y ago
The question is why they chose Python for apparently performance sensitive problems in the first place. The fact that they're now bumping against a performance wall and can't get around it by rewriting is a huge drawback when choosing Python as the language in the first place. Awareness that organisations often end up in these problems should be considered when choosing the language for a green field project.
- draaglom 4y ago
  The decision likely got made before performance was a major constraint. As I understand it, Instagram is a python codebase and (guessing) may be the primary motivator here. Probably the first, say, 100 person-years of effort poured into the Instagram codebase were done at a time when product development velocity, not performance, was the main priority.
- xiphias2 4y ago
  Instagram wasn't performance sensitive at all. It was iteration sensitive: if the team hadn't done it, some other group of people would have sold a company for billions of dollars to Facebook.

ehutch79 4y ago

As someone who uses python in their day job, I agree with this. When you spend more time io bound, it’s not an issue. If your python code is choking on cpu time, a lower level language is probably more appropriate.

gpderetta 4y ago
As someone who uses python in their day job, "IO bound" is a lie that is perpetuated to cover for the failing of the language. IO is very fast these days and python has the capacity of choking the cpu even on genuinely io bound problems . For example, do you think that python is an appropriate language to write pylint?
- dekhn 4y ago
  By definition, an IO bound problem does not "choke the CPU", because at that point, it would be CPU-bound.
  - gpderetta 4y ago
    Sure, I'm claiming that many problems that would be easily IO bound on other languages, are in fact CPU bound in python.
    - ehutch79 4y ago
      Then use a different language? I thought I said that?
- ehutch79 4y ago
  Network DB access is what I'm thinking about, not reading less than a K from on disk file. Also, any external API is an example. To answer your question; "it depends". Is pylints speed an issue? If not, then it's fine, if it is, the sure write it in go or rust or whatever. Javascript has a great example of this in esbuild.

itsoktocry 4y ago

>Seriously, use a faster language Some of the best engineers at the world's leading tech companies need to "pick another language", obviously.