5 ms·

Thanks for this. Will Julia advocates every use honest benchmarks to make their language look good? I doubt it.

dr_zoidberg 5y ago
OP has an open pull request on his repo[0] where someone made basically this same change (different names for the variables, but same idea). According to OP, he tried this but it resulted in slower execution for him. I'm not sure he really followed what that PR says. In the case of the submitter, it says on their machine it gave a 2x speedup (smaller than my ~7x, but still significant). [0] https://github.com/00sapo/cython_list_test/pull/3 https://github.com/00sapo/cython_list_test/pull/3
- nuisance-bear 5y ago
  If you carefully read OP's response in the PR, he says he tried static typing make_list in addition to iterate_list. Your comment elides the resource allocation step. Like OP, I observed that static typing make_list yields little benefit. It's runtime is 3x the runtime of iterate_list. And that makes sense. I'm not sure why we'd expect Cython to speed up the allocation of large numbers of Python objects.
  - dr_zoidberg 5y ago
    But OP didn't type iterate_list either. Perhaps he saw little benefit on one and assumed the same would happen in the other. Cython's annotation mode (cython -a) doesn't really help either here, showing both OPs and optimized versions with a strong yellow color, indicating that they would be about the same, but compiling and trying both definitely shows the improvement. Another funny thing: making a pure python version that uses the sum() built-in results in code that runs 80% as fast as the cython version, but also gives lower error (e-8 versus e-4 error that gets accumulated when the numbers are summed up one by one).
anotheranonym 5y ago
1000% this. Multiple times I’ve encountered a Julia benchmark claiming to show its superiority in a task I routinely perform. And every time the pro Julia benchmark turned out to be total BS.
- neolog 5y ago
  I wonder if it's more common for that language than other languages. Probably not many people have enough expertise in many different languages to compare fairly across them.
- eigenspace 5y ago
  It's hard to say anything about your use-case without more information, but I will say that one thing is that the way people talk about julia often seems to give people a mistaken impression about how to attain it's performance claims. Namely, realizing these claims requires learning the language, and actually taking advantage of it's strengths rather than just writing 'python in julia'. It's very common for Python users to show up on the Discourse forum and complain that julia is slower than python and then show some code that's basically just Python code written in julia, including a huge proliferation of global variables, allocating huge amounts of temporary arrays, etc. There's also a huge spectrum of 'benchmark quality' out there. E.g. the benchmarks this HN post features seem pretty shitty and are not measuring anything interesting or useful as far as I can tell.
ChrisRackauckas 5y ago
Note that the OP is a Python user, not a Julia user. The Github profile is a bunch of Python packages and the Julia code wasn't even optimized (https://github.com/00sapo/cython_list_test/pull/5 https://github.com/00sapo/cython_list_test/pull/5). If this test says anything, it at least would say that a inexperienced Python user could pick up Julia and do pretty well, even if the code they write isn't great. I think the right thing to do is just to help this guy learn to code for performance a bit better: it'll be better for him and would bring some positivity. Even if it doesn't say that, bashing people who use Julia for a repository made by a Python user is a new level of HN trolling.

OP has an open pull request on his repo[0] where someone made basically this same change (different names for the variables, but same idea). According to OP, he tried this but it resulted in slower execution for him. I'm not sure he really followed what that PR says. In the case of the submitter, it says on their machine it gave a 2x speedup (smaller than my ~7x, but still significant). [0] https://github.com/00sapo/cython_list_test/pull/3 https://github.com/00sapo/cython_list_test/pull/3

nuisance-bear 5y ago
If you carefully read OP's response in the PR, he says he tried static typing make_list in addition to iterate_list. Your comment elides the resource allocation step. Like OP, I observed that static typing make_list yields little benefit. It's runtime is 3x the runtime of iterate_list. And that makes sense. I'm not sure why we'd expect Cython to speed up the allocation of large numbers of Python objects.
- dr_zoidberg 5y ago
  But OP didn't type iterate_list either. Perhaps he saw little benefit on one and assumed the same would happen in the other. Cython's annotation mode (cython -a) doesn't really help either here, showing both OPs and optimized versions with a strong yellow color, indicating that they would be about the same, but compiling and trying both definitely shows the improvement. Another funny thing: making a pure python version that uses the sum() built-in results in code that runs 80% as fast as the cython version, but also gives lower error (e-8 versus e-4 error that gets accumulated when the numbers are summed up one by one).

anotheranonym 5y ago

1000% this. Multiple times I’ve encountered a Julia benchmark claiming to show its superiority in a task I routinely perform. And every time the pro Julia benchmark turned out to be total BS.

neolog 5y ago
I wonder if it's more common for that language than other languages. Probably not many people have enough expertise in many different languages to compare fairly across them.
eigenspace 5y ago
It's hard to say anything about your use-case without more information, but I will say that one thing is that the way people talk about julia often seems to give people a mistaken impression about how to attain it's performance claims. Namely, realizing these claims requires learning the language, and actually taking advantage of it's strengths rather than just writing 'python in julia'. It's very common for Python users to show up on the Discourse forum and complain that julia is slower than python and then show some code that's basically just Python code written in julia, including a huge proliferation of global variables, allocating huge amounts of temporary arrays, etc. There's also a huge spectrum of 'benchmark quality' out there. E.g. the benchmarks this HN post features seem pretty shitty and are not measuring anything interesting or useful as far as I can tell.

ChrisRackauckas 5y ago

Note that the OP is a Python user, not a Julia user. The Github profile is a bunch of Python packages and the Julia code wasn't even optimized (https://github.com/00sapo/cython_list_test/pull/5 https://github.com/00sapo/cython_list_test/pull/5). If this test says anything, it at least would say that a inexperienced Python user could pick up Julia and do pretty well, even if the code they write isn't great. I think the right thing to do is just to help this guy learn to code for performance a bit better: it'll be better for him and would bring some positivity. Even if it doesn't say that, bashing people who use Julia for a repository made by a Python user is a new level of HN trolling.