4 ms·

In the same vein as what Linus is trying to do now at LTT, it would be useful if there was a "programming journalism" lab which builds "test benches" to create

• okl 5y ago
  Like pts that https://www.phoronix.com/ https://www.phoronix.com/ uses?
• gostsamo 5y ago
  Usually the setups and architectural choices are so different that it would make more sense for everyone to get a good stat on which functions they are calling and how much and to make a prediction on the individual stats of each function. This does not take into account caching and multithreaded scenarios, but the map can never be the territory.
• littlecranky67 5y ago
  There are these: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/
  • cies 5y ago
    These are some microbenchmarks at best. Still nice to know who wins, but not representative of any real work scenario.
    • Hnus 5y ago
      What would be representation of some real work scenarios? I think their test suite covers wide spectrum of operations enough so to be able to draw conclusions from them. https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#fortunes https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Proj...
      • fenring 5y ago
        Unfortunently, after looking at the .net core implementation of this benchmarks, I wouldn't trust it at all. The code is just overengineerd to perform best at benchmark - everything hardcoded, custom routing to cover 2-3 routes with minimal overhead etc. It has nothing in common with real world code.
      • The_rationalist 5y ago
        The rust and c++ implementations are only on the top 10 because they use PostgreSQL pipelining/batching
      • the_duke 5y ago
        The Techempower entries are heavily gaming the system. They often strip out framework functionality and hyper-optimise for the specific benchmark, including things like pre-allocating the exact amount of memory needed to serve the request, not doing route matching et all, etc. They are basically an exercise in "how clever can we be to win the benchmark" rather than a realistic portrait of real world performance.
        • fabian2k 5y ago
          at least for .NET the versions that strip out framework functionality are marked separately, though this part is not that easy to understand if you don't know about it. There are several .NET entries from very low-level without MVC and without ORM up to the full stack. But still, these benchmarks have their uses but there are a lot of caveats you need to consider when looking at the results.
• buryat 5y ago
  why someone would do it fo free? with journalism you have advertiser that sponsor investigations albeit undirectly
  • Dayshine 5y ago
    LTT is a perfect example of how you can do it without any particularly plausible conflicting sponsorships. Do you really believe that Merch sales and sponsorship by Squarespace, VPN providers, sys-admin software, etc influence their coverage of CPUs?
• Philip-J-Fry 5y ago
  I think people should just bench their own code and measure their own performance increases.
  • deleted 5y ago
    [deleted]
• GuB-42 5y ago
  It is hard to do relevant tests of which language is the fastest. Really, writing fast code is mostly down to the programmer. For example C is widely recognized as the fastest non-assembly language simply because it leaves a lot to the programmer, C won't magically make your terrible code fast, unless you are using time-to-segfault as a metric. Assembly is the fastest if you know what you are doing, very few know what they are doing. So, what kind of code are you going to use for your benchmark? Highly optimized code written by experts spending way too much time, the "most idiomatic" code, code written by an average skilled programmer picked at random, code extracted from a big open source project? This can drastically change the ranking, so which one is the most relevant? If you go with the "most idiomatic" for instance, you miss out on the idea that parts can be optimized if needed, and that in real life, programmers aren't perfect and can write suboptimal code by mistake. There is also a cultural aspect to languages that may not be caught in benchmarks. For example, C programmers tend to have a culture of performance, they tend to know about their hardware, will try to save memory, make data structure efficient, etc... Python programmers, not so much, instead they tend to value readability and development time. You can't test languages like you test CPUs for instance. With CPUs, you just run the same code on them and time them. You can't do that for obvious reasons: your C compiler won't accept your Python code, it is necessarily an apples to oranges comparison.
  • kaba0 5y ago
    > Assembly is the fastest if you know what you are doing, very few know what they are doing. Just a nitpick but for any reasonably sized code, no. While some people can indeed do impressive optimizations on small segments of assembly, they are humans and they will fail to do trivial optimizations that are reliably done by compilers.
  • DeathArrow 5y ago
    In general the speed is lost because we make the CPU consume cycles to do things that are not necessary in order to solve a particular problem: -unneeded allocations -boxing/unboxing -garbage collection -interpreting
    • kaba0 5y ago