5 ms·

I'm always shocked at how much performance we leave on the table in these places. I did a super quick benchmark in python[0] and go [1] (based on necovek's[2] c

I'm always shocked at how much performance we leave on the table in these places. I did a super quick benchmark in python[0] and go [1] (based on necovek's[2] comment) and ran them both locally. The go implementation runs the entire benchmark quicker than the fastest of the python ones. The deviation in go's performance is still large, but far less so than Python's. Making the "wrong" choice for a single function call (bearing in mind that this is 10k iterations so we're still in the realm of scales even a moderate app can hit) in python is catastrophic, making the wrong choice for go is a significant slowdown but still 5x faster than doing it in Python. That sort of mental overhead is going to be everywhere, and it certainly doesn't encourage me to want to use python for a project. [0] https://www.online-python.com/9gcpKLe458 https://www.online-python.com/9gcpKLe458 [1] https://go.dev/play/p/zYKE0oZMFF4?v=goprev https://go.dev/play/p/zYKE0oZMFF4?v=goprev [2] https://news.ycombinator.com/item?id=41196915 https://news.ycombinator.com/item?id=41196915

serjester 2y ago
Cpython deliberately chooses to prioritize simplicity, stability and easy C extensions over raw speed. If you run the benchmark in PyPy (the performance optimized implementation) it’d be 10X faster. You could argue anything that’s performance bottlenecked shouldn’t be implemented in Python in the first place and therefore C compatibility is critical.
- maccard 2y ago
  I don't have easy access to pypy so I can't run the benchmark on that I'm afraid. I'd love to see some relative results though. > You could argue anything that’s performance bottlenecked shouldn’t be implemented in Python in the first place and therefore C compatibility is critical. Honestly, I think I'm heading towards "python should only be used for glue where performance does not matter". The sheer amount of time lost to waiting for slow runtimes is often brought up with node and electron, but I ran the benchmark in node and it's about halfway between go and python.
  - igouy 2y ago
    CPython 3.12.3 vs Node.js v22.2.0 https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python3-node.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
  - camdenreslink 2y ago
    Node isn't the slow part of electron. Node is actually pretty darn fast.
    - pjmlp 2y ago
      Indeed, shipping a full browser is.
bongodongobob 2y ago
Prototype in Python so you can get your ideas on the page then rewrite in something else has been my method for a while now.
- pjmlp 2y ago
  I rather do it in a language that has JIT in the box, or even AOT as alternative, then I hardly bother with rewrites. Something I learned by rewriting too much Tcl into C.
sgarland 2y ago
Write it in Python, profile it, and move the bottlenecks into C, called with ctypes.
- bbkane 2y ago
  If you're going to rewrite large chunks anyway, I'd use Go from the start.
  - carlmr 2y ago
    This is something I find quite fascinating. I've seen so much time wasted on Python this way it's incredible. Insanely smart optimizations done. But then you rewrite the whole thing in vanilla Rust without profiling once and it's still 20x faster, and much more readable than the mix of numpy vectorization and C-routines. It's kind of knowingly entering a sunk-cost fallacy for convenience. Before the cost is even sunk. Python is often just the wrong choice. And it's not even that much simpler if you ask me. In a language where you use value-based errors, have powerful sum types and strong typing guarantees, you can often get there faster, too IMO, because you can avoid a lot of mistakes by using the strictness of the compiler in your favor.
    - lpapez 2y ago
      > Insanely smart optimizations done. But then you rewrite the whole thing in vanilla Rust without profiling once and it's still 20x faster, and much more readable than the mix of numpy vectorization and C-routines. I'd argue that if you can rewrite the whole thing in Rust (or any other language), then it's not really a large project and it doesn't matter what you originally wrote it in. On multiple occassions I had to port parts of Python projects to C++ for performance reasons, and the amount of code you have to write (and time you need to spend) is mind blowing. Sometimes a single line of Python would expand to several files of C++, and you always had to redesign it somehow so that memory and lifetime management becomes feasible. Python is often the right choice IMO, because most of the time the convenience you mention trumps all other concerns, including performance. I bet you would get tons of examples from people if you "Ask HN" about "how many quickly written Python prototypes are still out there years later because their C++ replacement could never be implemented on time?"
heavyset_go 2y ago
Sounds about right, you can get a ~4x speed up by compiling Python with Nuitka or Mypyc.
rand_r 2y ago
Development speed and conciseness is what sells Python. Something simple like being able to do `breakpoint()` anywhere and get an interactive debugger is unmatched in Go.
- randomdata 2y ago
  Development speed in Go blows Python out of the water when approaching the project as an engineering project, but Python has it beat when approaching the project as a computer science project. And therein lies the rub for anything that is pushing the modernity of computer science (AI/ML, DSP, etc.) When born in the computer science domain, much of the preexisting work able to be leveraged is going to be built around Python, not Go (or any other language, for that matter), and it takes a lot of work to bring that to another system. It is pretty clear that just about anything other than Python is used in engineering when the science end is already well explored and the aforementioned work has already been done, but the available libraries are Python's selling feature otherwise.
- pjmlp 2y ago
  Yeah, but Go isn't the only alterantive, and something like `breakpoint()` anywhere goes back to Smalltalk and Lisp, with performance Python doesn't offer.
- igouy 2y ago
  Back in the day: https://cuis-smalltalk.github.io/TheCuisBook/Halt_0021.html https://cuis-smalltalk.github.io/TheCuisBook/Halt_0021.html
- maccard 2y ago
  Go's compile times are comparable to a decent sized python app's startup time. > Something simple like being able to do `breakpoint()` anywhere and get an interactive debugger is unmatched in Go. As opposed to using a debugger and setting a break and/or tracepoint, which you can do to a running program?
  - necovek 2y ago
    In a Python debugger, you can trivially augment the state of the program by executing Python code right there while debugging. I don't remember what you can do with Go's debugger, but generally for compiled programs and GDB, you can do a lot, but you can't really execute snippets of new code in-line. The simplicity is in the fact that Python debugger syntax is really mostly Python syntax, so you already know that if you've written the Python program you are debugging.

serjester 2y ago

Cpython deliberately chooses to prioritize simplicity, stability and easy C extensions over raw speed. If you run the benchmark in PyPy (the performance optimized implementation) it’d be 10X faster. You could argue anything that’s performance bottlenecked shouldn’t be implemented in Python in the first place and therefore C compatibility is critical.

maccard 2y ago
I don't have easy access to pypy so I can't run the benchmark on that I'm afraid. I'd love to see some relative results though. > You could argue anything that’s performance bottlenecked shouldn’t be implemented in Python in the first place and therefore C compatibility is critical. Honestly, I think I'm heading towards "python should only be used for glue where performance does not matter". The sheer amount of time lost to waiting for slow runtimes is often brought up with node and electron, but I ran the benchmark in node and it's about halfway between go and python.
- igouy 2y ago
  CPython 3.12.3 vs Node.js v22.2.0 https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python3-node.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- camdenreslink 2y ago
  Node isn't the slow part of electron. Node is actually pretty darn fast.
  - pjmlp 2y ago
    Indeed, shipping a full browser is.

bongodongobob 2y ago

Prototype in Python so you can get your ideas on the page then rewrite in something else has been my method for a while now.

pjmlp 2y ago
I rather do it in a language that has JIT in the box, or even AOT as alternative, then I hardly bother with rewrites. Something I learned by rewriting too much Tcl into C.

sgarland 2y ago

Write it in Python, profile it, and move the bottlenecks into C, called with ctypes.

bbkane 2y ago
If you're going to rewrite large chunks anyway, I'd use Go from the start.
- carlmr 2y ago
  This is something I find quite fascinating. I've seen so much time wasted on Python this way it's incredible. Insanely smart optimizations done. But then you rewrite the whole thing in vanilla Rust without profiling once and it's still 20x faster, and much more readable than the mix of numpy vectorization and C-routines. It's kind of knowingly entering a sunk-cost fallacy for convenience. Before the cost is even sunk. Python is often just the wrong choice. And it's not even that much simpler if you ask me. In a language where you use value-based errors, have powerful sum types and strong typing guarantees, you can often get there faster, too IMO, because you can avoid a lot of mistakes by using the strictness of the compiler in your favor.
  - lpapez 2y ago
    > Insanely smart optimizations done. But then you rewrite the whole thing in vanilla Rust without profiling once and it's still 20x faster, and much more readable than the mix of numpy vectorization and C-routines. I'd argue that if you can rewrite the whole thing in Rust (or any other language), then it's not really a large project and it doesn't matter what you originally wrote it in. On multiple occassions I had to port parts of Python projects to C++ for performance reasons, and the amount of code you have to write (and time you need to spend) is mind blowing. Sometimes a single line of Python would expand to several files of C++, and you always had to redesign it somehow so that memory and lifetime management becomes feasible. Python is often the right choice IMO, because most of the time the convenience you mention trumps all other concerns, including performance. I bet you would get tons of examples from people if you "Ask HN" about "how many quickly written Python prototypes are still out there years later because their C++ replacement could never be implemented on time?"

heavyset_go 2y ago

Sounds about right, you can get a ~4x speed up by compiling Python with Nuitka or Mypyc.

rand_r 2y ago

Development speed and conciseness is what sells Python. Something simple like being able to do `breakpoint()` anywhere and get an interactive debugger is unmatched in Go.

randomdata 2y ago
Development speed in Go blows Python out of the water when approaching the project as an engineering project, but Python has it beat when approaching the project as a computer science project. And therein lies the rub for anything that is pushing the modernity of computer science (AI/ML, DSP, etc.) When born in the computer science domain, much of the preexisting work able to be leveraged is going to be built around Python, not Go (or any other language, for that matter), and it takes a lot of work to bring that to another system. It is pretty clear that just about anything other than Python is used in engineering when the science end is already well explored and the aforementioned work has already been done, but the available libraries are Python's selling feature otherwise.
pjmlp 2y ago
Yeah, but Go isn't the only alterantive, and something like `breakpoint()` anywhere goes back to Smalltalk and Lisp, with performance Python doesn't offer.
igouy 2y ago
Back in the day: https://cuis-smalltalk.github.io/TheCuisBook/Halt_0021.html https://cuis-smalltalk.github.io/TheCuisBook/Halt_0021.html
maccard 2y ago
Go's compile times are comparable to a decent sized python app's startup time. > Something simple like being able to do `breakpoint()` anywhere and get an interactive debugger is unmatched in Go. As opposed to using a debugger and setting a break and/or tracepoint, which you can do to a running program?
- necovek 2y ago
  In a Python debugger, you can trivially augment the state of the program by executing Python code right there while debugging. I don't remember what you can do with Go's debugger, but generally for compiled programs and GDB, you can do a lot, but you can't really execute snippets of new code in-line. The simplicity is in the fact that Python debugger syntax is really mostly Python syntax, so you already know that if you've written the Python program you are debugging.