4 ms·

I think an important bit of context here is that computers are very, very good at speculative happy-path execution. The examples in the article seem gloomy: ho

by btown 1y ago

I think an important bit of context here is that computers are very, very good at speculative happy-path execution. The examples in the article seem gloomy: how could a JIT possibly do all the checks to make sure the arguments aren’t funky before adding them together, in a way that’s meaningfully better than just running the interpreter? But in practice, a JIT can create code that does these checks, and modern processors will branch-predict the happy path and effectively run it in parallel with the checks. JavaScript, too, has complex prototype chains and common use of boxed objects - but v8 has made common use cases extremely fast. I’m excited for the future of Python.

DanielHB 1y ago
The main problem is when the optimizations silently fail because of seemingly innocent changes and suddenly your performance tanked 10x. This is a problem with any language really (CPU cache misses are a thing afterall and many non-dynamic languages have boxed objects) but it is a much, much worse in dynamic languages like Python, JS and Ruby. Most of the time it doesn't matter, most high-throughput python code just invokes C/C++ where these concerns are not as big of a problem. Most JS code just invokes C/C++ browser DOM objects. As long as the hot-path is not in those languages you are not at such high risk of "innocent change tanked performance" Even server-side most JS/Python/Ruby code is just simple HTTP stack handlers and invoking databases and shuffling data around. And often large part of the process of handling a request (encoding JSON/XML/etc, parsing HTTP messages, etc) can be written in lower-level languages.
nxobject 1y ago
To be slightly flip, we could say that the Lisp Machine CISC-supports-language full stack design philosophy lives on in how massive M-series reorder buffers and ILP supports JavaScriptCore.
jerf 1y ago
That makes it so that in absolute terms, Python is not as slow as you might naively expect. But we don't measure programming language performance in absolute terms. We measure them in relative terms, generally against C. And while your Python code is speculating about how this Python object will be unboxed, where its methods are, how to unbox its parameters, what methods will be called on those, etc., compiled code is speculating on actual code the programmer has written, running that in parallel, such that by the time the Python interpreter is done speculating successfully on how some method call will resolve with actual objects the compiled code language is now done with ~50 lines of code of similar grammatical complexity. (Which is a sloppy term, since this is a bit of a sloppy conversation, but consider a series "p.x = y"-level statements in Python versus C as the case I'm looking at here.) There's no way around it. You can spend your amazingly capable speculative parallel CPU on churning through Python interpretation or you can spend it on doing real work, but you can't do both. After all, the interpreter is just C code too. It's not like it gets access to special speculation opcodes that no other program does.
- Demiurge 1y ago
  I love this “real work”. Real work, like writing linked lists, array bounds checking, all the error handling for opening files, etc, etc? There is a reason Python and C both have a use case, and it’s obvious Python will never be as fast as C doing “1 + 1”. The real “real work” is in getting stuff done, not just making sure the least amount of cpu cycles are used to accomplish some web form generation. Anyway, I think you’re totally right, in your general message. Python will never be the fastest language in all contexts. Still, there is a lot of room for optimization, and given it’s a popular language, it’s worth the effort.
  - btown 1y ago
    To put it another way, I choose Python because of its semantics around dynamic operator definition, duck typing etc. Just because I don’t write the bounds-checking and type-checking and dynamic-dispatch and error-handling code myself, doesn’t make it any less a conscious decision I made by choosing Python. It’s all “real work.”
    - kragen 1y ago
      Type checking and bounds checking aren't "real work" in the sense that, when somebody checks their bank account balance on your website or applies a sound effect to an audio track in their digital audio workstation, they don't think, "Oh good! The computer is going to do some type checking for me now!" Type checking and bounds checking may be good means to an end, but they are not the end, from the point of view of the outside world. Of course, the bank account is only a means to the end of paying the dentist for installing crowns on your teeth and whatnot, and the sound effect is only a means to the end of making your music sound less like Daft Punk or something, so it's kind of fuzzy. It depends on what people are thinking about achieving. As programmers, because we know the experience of late nights debugging when our array bounds overflow, we think of bounds checking and type checking as ends in themselves. But only up to a point! Often, type checking and bounds checking can be done at compile time, which is more efficient. When we do that, as long as it works correctly, we never† feel disappointed that our program isn't doing run-time type checks. We never look at our running programs and say, "This program would be better if it did more of its type checks at runtime!" No. Run-time type checking is purely a deadweight loss: wasting some of the CPU on computation that doesn't move the program toward achieving the goals we were trying to achieve when we wrote it. It may be a worthwhile tradeoff (for simplicity of implementation, for example) but we must weigh it on the debit side of the ledger, not the credit side. ______ † Well, unless we're trying to debug a PyPy type-specialization bug or something. Then we might work hard to construct a program that forces PyPy to do more type-checking at runtime, and type checking does become an end.
fpoling 1y ago
Although JS supports prototype mutations, the with operator and other constructs that make optimization harder, typical JS code does not use that. Thus JIT can add few checks for presence of problematic constructions to direct it to a slow path while optimizing not particularly big set of common patterns. And then the JS JIT does not need to care much about calling arbitrary native code as the browser internals can be adjusted/refactored to tune to JIT needs. With Python that does not work. There are simply more optimization-unfriendly constructs and popular libraries use those. And Python calls arbitrary C libraries with fixed ABI. So optimizing Python is inherently more difficult.
josefx 1y ago
> but v8 has made common use cases extremely fast. I’m excited for the future of Python. Isn't v8 still entirely single threaded with limited message passing? Python just went through a lot of work to make multithreaded code faster, it would be disappointing if it had to scrap threading entirely and fall back to multiprocessing on shared memory in order to match v8.
- zozbot234 1y ago
  Multithreaded code is usually bottlenecked by memory bandwidth, even more so than raw compute. C/C++/Rust are great at making efficient use of memory bandwidth, whereas scripting languages are rather wasteful of it by comparison. So I'm not sure that multithreading will do much to bridge the performance gap between binary compiled languages and scripting languages like Python.
- loeg 1y ago
  JS is single-threaded. Python isn't.
ratmeadow 1y ago
I wonder if branch prediction can still hide the performance loss when the happy path checks become large/complex. Branch prediction is a very low level optimisation. And if the predictor is right you don't get everything for free. The CPU must still evaluate the condition, which takes resources, albeit it's no longer on the critical path. However I'd think the CPU would stall if it got too far ahead of the condition execution (ultimately all the code must execute before the program completes). Perhaps given the nature of Python, the checks would be so complex that in a tight loop they'd exert significant resource pressure?
TheOtherHobbes 1y ago
There's an obvious answer - run everything on GPUs. Each speculative branch runs in parallel on its own core, and you add a layer of super-fast branch switching when a branch runs into a problem. Given the current state of computing, I am unable to state definitively if this suggestion is satire.

by btown 1y ago

DanielHB 1y ago

The main problem is when the optimizations silently fail because of seemingly innocent changes and suddenly your performance tanked 10x. This is a problem with any language really (CPU cache misses are a thing afterall and many non-dynamic languages have boxed objects) but it is a much, much worse in dynamic languages like Python, JS and Ruby. Most of the time it doesn't matter, most high-throughput python code just invokes C/C++ where these concerns are not as big of a problem. Most JS code just invokes C/C++ browser DOM objects. As long as the hot-path is not in those languages you are not at such high risk of "innocent change tanked performance" Even server-side most JS/Python/Ruby code is just simple HTTP stack handlers and invoking databases and shuffling data around. And often large part of the process of handling a request (encoding JSON/XML/etc, parsing HTTP messages, etc) can be written in lower-level languages.

nxobject 1y ago

To be slightly flip, we could say that the Lisp Machine CISC-supports-language full stack design philosophy lives on in how massive M-series reorder buffers and ILP supports JavaScriptCore.

jerf 1y ago

That makes it so that in absolute terms, Python is not as slow as you might naively expect. But we don't measure programming language performance in absolute terms. We measure them in relative terms, generally against C. And while your Python code is speculating about how this Python object will be unboxed, where its methods are, how to unbox its parameters, what methods will be called on those, etc., compiled code is speculating on actual code the programmer has written, running that in parallel, such that by the time the Python interpreter is done speculating successfully on how some method call will resolve with actual objects the compiled code language is now done with ~50 lines of code of similar grammatical complexity. (Which is a sloppy term, since this is a bit of a sloppy conversation, but consider a series "p.x = y"-level statements in Python versus C as the case I'm looking at here.) There's no way around it. You can spend your amazingly capable speculative parallel CPU on churning through Python interpretation or you can spend it on doing real work, but you can't do both. After all, the interpreter is just C code too. It's not like it gets access to special speculation opcodes that no other program does.

Demiurge 1y ago
I love this “real work”. Real work, like writing linked lists, array bounds checking, all the error handling for opening files, etc, etc? There is a reason Python and C both have a use case, and it’s obvious Python will never be as fast as C doing “1 + 1”. The real “real work” is in getting stuff done, not just making sure the least amount of cpu cycles are used to accomplish some web form generation. Anyway, I think you’re totally right, in your general message. Python will never be the fastest language in all contexts. Still, there is a lot of room for optimization, and given it’s a popular language, it’s worth the effort.
- btown 1y ago
  To put it another way, I choose Python because of its semantics around dynamic operator definition, duck typing etc. Just because I don’t write the bounds-checking and type-checking and dynamic-dispatch and error-handling code myself, doesn’t make it any less a conscious decision I made by choosing Python. It’s all “real work.”
  - kragen 1y ago
    Type checking and bounds checking aren't "real work" in the sense that, when somebody checks their bank account balance on your website or applies a sound effect to an audio track in their digital audio workstation, they don't think, "Oh good! The computer is going to do some type checking for me now!" Type checking and bounds checking may be good means to an end, but they are not the end, from the point of view of the outside world. Of course, the bank account is only a means to the end of paying the dentist for installing crowns on your teeth and whatnot, and the sound effect is only a means to the end of making your music sound less like Daft Punk or something, so it's kind of fuzzy. It depends on what people are thinking about achieving. As programmers, because we know the experience of late nights debugging when our array bounds overflow, we think of bounds checking and type checking as ends in themselves. But only up to a point! Often, type checking and bounds checking can be done at compile time, which is more efficient. When we do that, as long as it works correctly, we never† feel disappointed that our program isn't doing run-time type checks. We never look at our running programs and say, "This program would be better if it did more of its type checks at runtime!" No. Run-time type checking is purely a deadweight loss: wasting some of the CPU on computation that doesn't move the program toward achieving the goals we were trying to achieve when we wrote it. It may be a worthwhile tradeoff (for simplicity of implementation, for example) but we must weigh it on the debit side of the ledger, not the credit side. ______ † Well, unless we're trying to debug a PyPy type-specialization bug or something. Then we might work hard to construct a program that forces PyPy to do more type-checking at runtime, and type checking does become an end.

fpoling 1y ago

Although JS supports prototype mutations, the with operator and other constructs that make optimization harder, typical JS code does not use that. Thus JIT can add few checks for presence of problematic constructions to direct it to a slow path while optimizing not particularly big set of common patterns. And then the JS JIT does not need to care much about calling arbitrary native code as the browser internals can be adjusted/refactored to tune to JIT needs. With Python that does not work. There are simply more optimization-unfriendly constructs and popular libraries use those. And Python calls arbitrary C libraries with fixed ABI. So optimizing Python is inherently more difficult.

josefx 1y ago

> but v8 has made common use cases extremely fast. I’m excited for the future of Python. Isn't v8 still entirely single threaded with limited message passing? Python just went through a lot of work to make multithreaded code faster, it would be disappointing if it had to scrap threading entirely and fall back to multiprocessing on shared memory in order to match v8.

zozbot234 1y ago
Multithreaded code is usually bottlenecked by memory bandwidth, even more so than raw compute. C/C++/Rust are great at making efficient use of memory bandwidth, whereas scripting languages are rather wasteful of it by comparison. So I'm not sure that multithreading will do much to bridge the performance gap between binary compiled languages and scripting languages like Python.
loeg 1y ago
JS is single-threaded. Python isn't.

ratmeadow 1y ago

I wonder if branch prediction can still hide the performance loss when the happy path checks become large/complex. Branch prediction is a very low level optimisation. And if the predictor is right you don't get everything for free. The CPU must still evaluate the condition, which takes resources, albeit it's no longer on the critical path. However I'd think the CPU would stall if it got too far ahead of the condition execution (ultimately all the code must execute before the program completes). Perhaps given the nature of Python, the checks would be so complex that in a tight loop they'd exert significant resource pressure?

TheOtherHobbes 1y ago

There's an obvious answer - run everything on GPUs. Each speculative branch runs in parallel on its own core, and you add a layer of super-fast branch switching when a branch runs into a problem. Given the current state of computing, I am unable to state definitively if this suggestion is satire.