4 ms·

Yet another "X faster than Y" where inside X all hot jobs are done by really good-tuned C or C++. Facepalm. Without real problem-solving (business logic) writt

by cyber1 4y ago

Yet another "X faster than Y" where inside X all hot jobs are done by really good-tuned C or C++. Facepalm. Without real problem-solving (business logic) written in Python, this is only lightweight wrapper on C/C++. When the number of Python code start growing in the hot path, then these blazing-amazing thruput numbers tremendously will be going down.

inglor 4y ago
I'm so happy this is (currently) the top comment and people are starting to realize measuring perf with these well tuned micro-benchmarks is a sham.
- jwdunne 4y ago
  Yes. I was thinking exactly this when coming into the comments. Real world benchmarks take more time to prepare, we get that, but let’s be honest: a bold claim _needs_ bold evidence.
- warinukraine 4y ago
  Why is it a sham? It's useful to know that x is faster. As a user of x I don't really care if the reason it's faster is it's C++ under the hood. That's really an implementation detail for me.
  - lupire 4y ago
    What happens when you use X because it's fast at one thing in a microbenchmark, but it locks you into an environment that is slow at the rest of your program?
  - inglor 4y ago
    Because it's measuring a scenario that's not representative of real usages of X and is usually tweaked for the framework/tool the author is trying to showcase. For example: all the deno/bun benchmarks often use the Node.js frameworks in an unnecessarily slow way and don't measure apples to apples. (like using uWS in bun but not uWS for Node in the benchmark). This one in particular just measures the speed of "hello world" in plaintext and JSON with very specific parameters.
  - anonymoushn 4y ago
    The real thing will be some mix of your code and the library's code, and in cases like this where you'd be writing all your logic in Python vs Golang, that will have a significant effect on the results. A while back I implemented a game rules engine and MCTS in Torch (the Lua library). The training loop spent like half of its time running the rules engine. Writing it in Python would have been a disaster in comparison. To really make good use of the hardware and spend more time in the optimized machine learning code provided by libraries, one would have to write their rules engine in some language other than Lua or Python.
  - fsdjkflsjfsoij 4y ago
    Because the moment you actually write any server or business logic in Python performance drops off a cliff and it doesn't differ significantly from any of the other Python web frameworks.
Waterluvian 4y ago
I think this is correct and does demonstrate that these comparisons are not all that useful. But I also think that writing Python and then dropping down into C or C++ for performance is perfectly valid and often a great idea. Of course, there's nothing wrong with just using a different language that's a sensible middle ground between Python and C (like Go). But hold on to the baby when that bathwater is being dumped: there's nothing wrong with writing a blend of Python and C++ for real uses.
commitpizza 4y ago
> Yet another "X faster than Y" where inside X all hot jobs are done by really good-tuned C or C++. Facepalm. This is the case for basically every programming language or performant library that exists. Yet, I never see this when discussions exists about node libraries or other languages for that matter. I mean you could argue that node itself is just a thin wrapper around fast C++ libraries. Who the fuck cares if the request goes down to some compiled C++ library? I still write my logic in Python and get this benefit anyway. This is what makes it great.
- charles_f 4y ago
  > Who the fuck cares if the request goes down to some compiled C++ library? The way this is titled, you know someone will read this as "Python can be faster than go" and influence their decision when it comes to performance ; that's actually why I clicked on that, thinking "how the hell did they achieve faster speed with Python".
  - commitpizza 4y ago
    Well, it could be true IMHO. If I run my code that calls down to some ultra fast, mega well done C++ library that is much better than anything in the Go community my code then completes requests faster than the Go alternative. If that is the case, that should influence the decision when it comes to performance. If performance is a very important deal in your app, you have to test this yourself so that you reach the performance you need. Perhaps you can write C++ extensions to Python for the perf-heavy parts yourself instead of having to write the entire application in a language that takes much longer time to develop in?
- anthonygd 4y ago
  > Yet, I never see this when discussions exists about node libraries or other languages for that matter. Huh? It's the first comment for pretty much any of these stupid performance comparisons. Does someone really think node performance is great, or pick node because of its performance? > I still write my logic in Python and get this benefit anyway. That's part of the point. Writing your logic in Python is fine, but then your app performance isn't winning any comparisons. No one cares about the performance this article is explicitly talking about.
carlsborg 4y ago
Would golang interfaced with the C++ implemenation be as fast? IMO one of the nice features of Python is being able to seamlessly drop to numba, or a C++ module when you need an optimized fastpath, but the rest of your application, the bulk of it, need not.
deleted 4y ago
[deleted]

by cyber1 4y ago

inglor 4y ago

I'm so happy this is (currently) the top comment and people are starting to realize measuring perf with these well tuned micro-benchmarks is a sham.

jwdunne 4y ago
Yes. I was thinking exactly this when coming into the comments. Real world benchmarks take more time to prepare, we get that, but let’s be honest: a bold claim _needs_ bold evidence.
warinukraine 4y ago
Why is it a sham? It's useful to know that x is faster. As a user of x I don't really care if the reason it's faster is it's C++ under the hood. That's really an implementation detail for me.
- lupire 4y ago
  What happens when you use X because it's fast at one thing in a microbenchmark, but it locks you into an environment that is slow at the rest of your program?
- inglor 4y ago
  Because it's measuring a scenario that's not representative of real usages of X and is usually tweaked for the framework/tool the author is trying to showcase. For example: all the deno/bun benchmarks often use the Node.js frameworks in an unnecessarily slow way and don't measure apples to apples. (like using uWS in bun but not uWS for Node in the benchmark). This one in particular just measures the speed of "hello world" in plaintext and JSON with very specific parameters.
- anonymoushn 4y ago
  The real thing will be some mix of your code and the library's code, and in cases like this where you'd be writing all your logic in Python vs Golang, that will have a significant effect on the results. A while back I implemented a game rules engine and MCTS in Torch (the Lua library). The training loop spent like half of its time running the rules engine. Writing it in Python would have been a disaster in comparison. To really make good use of the hardware and spend more time in the optimized machine learning code provided by libraries, one would have to write their rules engine in some language other than Lua or Python.
- fsdjkflsjfsoij 4y ago
  Because the moment you actually write any server or business logic in Python performance drops off a cliff and it doesn't differ significantly from any of the other Python web frameworks.

Waterluvian 4y ago

I think this is correct and does demonstrate that these comparisons are not all that useful. But I also think that writing Python and then dropping down into C or C++ for performance is perfectly valid and often a great idea. Of course, there's nothing wrong with just using a different language that's a sensible middle ground between Python and C (like Go). But hold on to the baby when that bathwater is being dumped: there's nothing wrong with writing a blend of Python and C++ for real uses.

commitpizza 4y ago

> Yet another "X faster than Y" where inside X all hot jobs are done by really good-tuned C or C++. Facepalm. This is the case for basically every programming language or performant library that exists. Yet, I never see this when discussions exists about node libraries or other languages for that matter. I mean you could argue that node itself is just a thin wrapper around fast C++ libraries. Who the fuck cares if the request goes down to some compiled C++ library? I still write my logic in Python and get this benefit anyway. This is what makes it great.

charles_f 4y ago
> Who the fuck cares if the request goes down to some compiled C++ library? The way this is titled, you know someone will read this as "Python can be faster than go" and influence their decision when it comes to performance ; that's actually why I clicked on that, thinking "how the hell did they achieve faster speed with Python".
- commitpizza 4y ago
  Well, it could be true IMHO. If I run my code that calls down to some ultra fast, mega well done C++ library that is much better than anything in the Go community my code then completes requests faster than the Go alternative. If that is the case, that should influence the decision when it comes to performance. If performance is a very important deal in your app, you have to test this yourself so that you reach the performance you need. Perhaps you can write C++ extensions to Python for the perf-heavy parts yourself instead of having to write the entire application in a language that takes much longer time to develop in?
anthonygd 4y ago
> Yet, I never see this when discussions exists about node libraries or other languages for that matter. Huh? It's the first comment for pretty much any of these stupid performance comparisons. Does someone really think node performance is great, or pick node because of its performance? > I still write my logic in Python and get this benefit anyway. That's part of the point. Writing your logic in Python is fine, but then your app performance isn't winning any comparisons. No one cares about the performance this article is explicitly talking about.

carlsborg 4y ago

Would golang interfaced with the C++ implemenation be as fast? IMO one of the nice features of Python is being able to seamlessly drop to numba, or a C++ module when you need an optimized fastpath, but the rest of your application, the bulk of it, need not.

deleted 4y ago

[deleted]