4 ms·

It’s comparing unassembled assembler to JITed code and compilers that are pulling in precompiled libraries. I feel it needs some kind of normalizing. I get tha

It’s comparing unassembled assembler to JITed code and compilers that are pulling in precompiled libraries. I feel it needs some kind of normalizing. I get that it is illustrating bloat but it doesn’t really illustrate where that’s coming from. Maybe only the output of the JITs should counts, or the syscalls required to assemble the example should be included. Are musl and glibc really wasting cycles or are they doing something that the example is missing. Fun to think about.

ddevault 7y ago
It's deliberate that JITs, interpreters, and compiled languages are compared on the same terms here. JITs and interpreters are fundamentally less performant than compiled languages, they don't get a pass on performance tests just because it's by design.
- joshuamorton 7y ago
  > JITs This is...highly context dependent. For highly polymorphic code, my understanding is that JITs can outperform precompiled binaries, since they can inline virtual/polymorphic calls in tight loops. This also isn't a "performance" test in any real sense. It's a test of startup time. Where, yes, JITs lose, but unless you're writing short lived interactive command line tools, or something that runs on lambda, that shouldn't be a concern. For "normal" serverside or desktop apps that run for more than, say, 30 seconds, the difference between 0s of startup and 0.2s of startup time is literally in the noise.
  - ddevault 7y ago
    Startup time is the least interesting metric in this article. The more interesting metric is the number of syscalls. This isn't a measure of performance, it's a measure of complexity and busywork. Complexity tends to indirectly affect performance, but that's not the point of the article.
    - joshuamorton 7y ago
      Complexity of what? The resulting generated binary? Well no, a python binary is smaller than the c binary. The toolchain? Well gcc is pretty complex and that's unaccounted for. The build process? Again, no. The closest thing I can think of is the language runtime. But why do I care about how complex the language runtime is? Often more complex language runtimes make my life easier anyway, and they're all sitting atop the Intel microcode magic box anyway. There's a very specific definition of complexity you're using, and I'm still not sure what it is. In my world, you usually add complexity to eek out extra performance by breaking the less complex abstractions.
      - ddevault 7y ago
        The compleixty of the system.
        joshuamorton 7y ago
        I'm still confused. Why is runtime compilation a component of the "system", but aot compilation is not? Why are python interpreters a component of the system, but microcode interpreters and aot compilation not? If you haven't given a clear definition of what "the system" is, I can't really use your evaluation to influence my decision making.
        ddevault 7y ago
        Because AOT compilation can be used to construct a system which is functional without the compiler, but runtime compilation requires it at runtime. The difference is plainly obvious.
        cycloptic 7y ago
        That difference is irrelevant when your production system is updated a lot, or when you actually need the runtime compilation, which a lot of programs do. Even on mobile and embedded where you would think this wasn't true, the customers still really don't care if given the luxury to do so by a fast CPU and high capacity battery.
tensor 7y ago
It's not even meaningfully illustrating bloat. Hello world is an unrealistic edge case. Any program that does something useful will be far more complicated, and it's entirely possible that a lot of the extra stuff being measured here will required anyways.
zamadatix 7y ago
> It’s comparing unassembled assembler to JITed code No, he runs the assembly through NASM + GCC as documented on the page. I think it's a comparison of "when the user runs the program what runs, how long does it take and how much disk space does it need" based of the column headings. It's not a comparison of the tooling prior to the user's computer as far as I can tell.

ddevault 7y ago

It's deliberate that JITs, interpreters, and compiled languages are compared on the same terms here. JITs and interpreters are fundamentally less performant than compiled languages, they don't get a pass on performance tests just because it's by design.

joshuamorton 7y ago
> JITs This is...highly context dependent. For highly polymorphic code, my understanding is that JITs can outperform precompiled binaries, since they can inline virtual/polymorphic calls in tight loops. This also isn't a "performance" test in any real sense. It's a test of startup time. Where, yes, JITs lose, but unless you're writing short lived interactive command line tools, or something that runs on lambda, that shouldn't be a concern. For "normal" serverside or desktop apps that run for more than, say, 30 seconds, the difference between 0s of startup and 0.2s of startup time is literally in the noise.
- ddevault 7y ago
  Startup time is the least interesting metric in this article. The more interesting metric is the number of syscalls. This isn't a measure of performance, it's a measure of complexity and busywork. Complexity tends to indirectly affect performance, but that's not the point of the article.
  - joshuamorton 7y ago
    Complexity of what? The resulting generated binary? Well no, a python binary is smaller than the c binary. The toolchain? Well gcc is pretty complex and that's unaccounted for. The build process? Again, no. The closest thing I can think of is the language runtime. But why do I care about how complex the language runtime is? Often more complex language runtimes make my life easier anyway, and they're all sitting atop the Intel microcode magic box anyway. There's a very specific definition of complexity you're using, and I'm still not sure what it is. In my world, you usually add complexity to eek out extra performance by breaking the less complex abstractions.
    - ddevault 7y ago
      The compleixty of the system.
      - joshuamorton 7y ago
        I'm still confused. Why is runtime compilation a component of the "system", but aot compilation is not? Why are python interpreters a component of the system, but microcode interpreters and aot compilation not? If you haven't given a clear definition of what "the system" is, I can't really use your evaluation to influence my decision making.
        ddevault 7y ago
        Because AOT compilation can be used to construct a system which is functional without the compiler, but runtime compilation requires it at runtime. The difference is plainly obvious.
        cycloptic 7y ago
        That difference is irrelevant when your production system is updated a lot, or when you actually need the runtime compilation, which a lot of programs do. Even on mobile and embedded where you would think this wasn't true, the customers still really don't care if given the luxury to do so by a fast CPU and high capacity battery.

tensor 7y ago

It's not even meaningfully illustrating bloat. Hello world is an unrealistic edge case. Any program that does something useful will be far more complicated, and it's entirely possible that a lot of the extra stuff being measured here will required anyways.

zamadatix 7y ago

> It’s comparing unassembled assembler to JITed code No, he runs the assembly through NASM + GCC as documented on the page. I think it's a comparison of "when the user runs the program what runs, how long does it take and how much disk space does it need" based of the column headings. It's not a comparison of the tooling prior to the user's computer as far as I can tell.