4 ms·

It's interesting to imagine the head scratching going on in this thread. All these clever people, stumped. 'How can this be!???'. Quite the post, bravo OP. But

It's interesting to imagine the head scratching going on in this thread. All these clever people, stumped. 'How can this be!???'. Quite the post, bravo OP. But seriously, other comments have this nailed. This 'benchmark' isn't comparing apples with apples. There are too many quirks, optimisations and design decisions (of a compiler) that have not being taken into account. Also, as others have repeated, this isn't indicative of actual usage - real world usage. In that regard, this type of 'benchmark' isn't scientific at all, and worse, is misleading a naive reader.

Dylan16807 2y ago
While it's not (necessarily) indicative of the languages as a whole, it looks pretty apples to apples to me. These are very simple programs with no fancy tricks going on.
- hatthew 2y ago
  It's comparing a granny smith apple to a mcintosh apple by hitting them with a baseball bat and inferring the texture of the apple from that. It uses them in a way that almost nobody does in real life, and is a vague proxy to only one of several important factors. If you use this test as a basis for deciding which you would like, you're going to have a bad time, whereas you'll be fine if you merely look at what other people like.
  - Dylan16807 2y ago
    Is the "way" it's using things affecting performance significantly? If not then it's a pretty good proxy to one or two dimensions of performance, and a mix of similar benchmarks could paint a good fraction of a reasonable picture.
    - hatthew 2y ago
      If you want to buy a bushel of apples and hit them with an array of objects at varying velocities and then try to construct a model that can use the resulting splatter patterns to infer the texture of the apple as experienced by humans, you're welcome to do so. Personally, I'm going to bite the apple.
      - Dylan16807 2y ago
        I do not accept your analogy. This program is a perfectly reasonable "bite". It's not doing weird stuff.
        hatthew 2y ago
        The program is a useless toy that nobody would write and expect to be performant in real life. I can write an optimized version of the algorithm that prints the exact same output (barring edge cases) without looping. With this new algorithm (really just a single expression), the C program is 500x faster than before and the python script is 5000x faster than before. So python is only 10x slower than C in this example (and I'd imagine that this is measuring more overhead and less arithmetic). Presumably the C compiler has a relative advantage at optimizing the looped algorithm vs the arithmetic algorithm. How much? Is C 100x faster than python, or was the compiler able to get closer to the non-looping algorithm than python? How close? Is it something about the arithmetic or the looping that's natively faster in C? What part of it was optimized by the compiler? Since nobody does large loops in python, would python be more comparable if we introduced a C-backed library like numpy? What other objects do you think we should bash this apple with?
        Dylan16807 2y ago
        Do you think a slightly more complex piece of math that can't be replaced with an equation would have significantly different performance from the benchmark? I don't. So the simple math just makes things easier, it doesn't harm results. Your suggestion to skip the math entirely is the apple-smash. > Since nobody does large loops in python Oh I doubt that. Numpy would be fine as a different entry.
        hatthew 2y ago
        I don't know whether or not a more complex piece of math would change the outcome, so I would err on the side of not making assumptions until I had done more benchmarks and investigated the inner workings more. You probably know more about C than I do and might be able to make that assumption more confidently, but I think most people know less than either of us, and will read too much into a single statistic without context. Nobody who cares about performance enough to look at a benchmark like this would do large loops in pure python.

Dylan16807 2y ago

While it's not (necessarily) indicative of the languages as a whole, it looks pretty apples to apples to me. These are very simple programs with no fancy tricks going on.

hatthew 2y ago
It's comparing a granny smith apple to a mcintosh apple by hitting them with a baseball bat and inferring the texture of the apple from that. It uses them in a way that almost nobody does in real life, and is a vague proxy to only one of several important factors. If you use this test as a basis for deciding which you would like, you're going to have a bad time, whereas you'll be fine if you merely look at what other people like.
- Dylan16807 2y ago
  Is the "way" it's using things affecting performance significantly? If not then it's a pretty good proxy to one or two dimensions of performance, and a mix of similar benchmarks could paint a good fraction of a reasonable picture.
  - hatthew 2y ago
    If you want to buy a bushel of apples and hit them with an array of objects at varying velocities and then try to construct a model that can use the resulting splatter patterns to infer the texture of the apple as experienced by humans, you're welcome to do so. Personally, I'm going to bite the apple.
    - Dylan16807 2y ago
      I do not accept your analogy. This program is a perfectly reasonable "bite". It's not doing weird stuff.
      - hatthew 2y ago
        The program is a useless toy that nobody would write and expect to be performant in real life. I can write an optimized version of the algorithm that prints the exact same output (barring edge cases) without looping. With this new algorithm (really just a single expression), the C program is 500x faster than before and the python script is 5000x faster than before. So python is only 10x slower than C in this example (and I'd imagine that this is measuring more overhead and less arithmetic). Presumably the C compiler has a relative advantage at optimizing the looped algorithm vs the arithmetic algorithm. How much? Is C 100x faster than python, or was the compiler able to get closer to the non-looping algorithm than python? How close? Is it something about the arithmetic or the looping that's natively faster in C? What part of it was optimized by the compiler? Since nobody does large loops in python, would python be more comparable if we introduced a C-backed library like numpy? What other objects do you think we should bash this apple with?
        Dylan16807 2y ago
        Do you think a slightly more complex piece of math that can't be replaced with an equation would have significantly different performance from the benchmark? I don't. So the simple math just makes things easier, it doesn't harm results. Your suggestion to skip the math entirely is the apple-smash. > Since nobody does large loops in python Oh I doubt that. Numpy would be fine as a different entry.
        hatthew 2y ago
        I don't know whether or not a more complex piece of math would change the outcome, so I would err on the side of not making assumptions until I had done more benchmarks and investigated the inner workings more. You probably know more about C than I do and might be able to make that assumption more confidently, but I think most people know less than either of us, and will read too much into a single statistic without context. Nobody who cares about performance enough to look at a benchmark like this would do large loops in pure python.