6 ms·

Dear Walter, Thank you for the great work you are doing. If you could get people to include D in the benchmarks / shootouts that get published it would help i

Dear Walter, Thank you for the great work you are doing. If you could get people to include D in the benchmarks / shootouts that get published it would help improve the popularity of D. FYI - Thatneedle.com already uses it for part of the backend data processing workflow.

WalterBright 8y ago
Isaac Gouy is the gatekeeper for the shootout, and he refuses to include D for reasons he's refused to elucidate. I stopped publishing benchmarks myself 15 years ago as people always assumed my thumb was on the scale.
- codyguy 8y ago
  I was a bit surprised by the inclusion of lesser languages and the omission of D !! Please share the secret of your tenacity and inspiration in the face of such "setbacks". How do you do it on a day to day basis?
  - WalterBright 8y ago
    I know how good D is and I have a lot of confidence in it. There are a lot of great people using it that are getting their jobs done faster & better and are thereby making more money. That's what matters to me. Everybody likes to make more money :-) Back when I was a brand new engineer at Boeing, I was talking to my lead engineer about the beauty of aerospace engineering. He smiled and said I didn't get it. Boeing wasn't making airplanes, they were building money making machines for airline companies. That the airplane turned out to be a beauty was just a happy side effect! D isn't about my personal aggrandizement. It's about how effective it is for users at solving their problems and making money for them.
- e12e 8y ago
  Has there been any recent discussion on this? I suspect earlier reasons might have included D not being packaged for Debian? It can be though to shake "early license impressions" - D (dmd specifically) is one example, Ada (FSF GNAT vs AdaCore's GNAT Pro and Ada-gnat - GPL w/o runtime exception) is another. [ed: if others are interested in "make your own measurements and host it yourself", relevant page with link to code etc appear to be: https://benchmarksgame-team.pages.debian.net/benchmarksgame/play.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... ]
  - WalterBright 8y ago
    > any recent discussion Not since I asked him to stop dropping by the D forum to remind us that he wouldn't include D in the shootout. I've personally lost all interest in it. (The benchmarks are also small and too easily gamed by the author of the benchmark code, and the compiler developer. I encourage people to run timing tests on their own code, as that is what matters to them.)
    - girvo 8y ago
      My experience has been that I get better ideas of benchmarks by profiling my real application code, porting the hotspots to the new language or framework within a mostly bare skeleton and testing it. It’s not perfect, but gives me a better idea as to whether moving will actually help!
    - petre 8y ago
      I did run my own prime number crunching benchmark for fun and D blew Rust away but lagged behind Go. I used mutable Vec and HashSet in Rust, associative arrays and arrays in D, maps and arrays in Go to store the sieves.
      - dvfjsdhgfv 8y ago
        You see, that's the point - even if you do something as simple, there are many ways to do it (and different compiler versions, especially in the case of D), different optimizations at the code and compiler level and all that - it's practically impossible to have a reliable, comprehensive benchmark. (That said, D is lightning fast for me!)
      - steveklabnik 8y ago
        HashSet uses a cryptographically secure hashing algorithm by default, it’s gonna be slow.
        petre 8y ago
        What's the Rust equivalent of Go maps or D associative arrays then?
- igouy 8y ago
  >> for reasons he's refused to elucidate For the simplest reasons that I have stated to you many times! Including every language implementation that the language author would like to have included is more work than I am willing to do, period. Sept 13 2008: 63 language implementations were shown- https://web.archive.org/web/20080913030117/http://shootout.alioth.debian.org:80/gp4/ https://web.archive.org/web/20080913030117/http://shootout.a... - currently, 27 language implementations are shown. You could truthfully say "he refuses to include [at-least 30 language implementations]". In that regard, there's nothing special about D.
  - exikyut 8y ago
    I have no context regarding the history of the benchmarks game. With that in mind, I have a question: What about you create a technical specification (including both technical depth and common-sense breadth) for what kind of benchmark you'll accept, throw the whole thing on GitHub, and then refuse 99% of pull requests? :) (ie, only accept really really good quality benchmark implementations) Eventually, enough developers unimpressed that Language X is not adequately represented would step up to the plate and maintain good-quality benchmark code. (This could get pretty interesting with rapidly-evolving languages like Rust.) Obviously this is all very ideal and I can see so many ways such an endeavor could go horribly wrong, sure. I can also very easily see you having floated such an idea then discarded it for reasons I haven't even thought of.
    - igouy 8y ago
      What about I work to publish crowd-sourced programs and measurement scripts: so that anyone can make their own comparisons, on their own hardware, against whatever other language implementations they want to write programs for :-) https://benchmarksgame-team.pages.debian.net/benchmarksgame/play.html#languagex https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
      - exikyut 8y ago
        Ah; that certainly works too! The only thing I wonder about is allowing any given set of hardware to be compared with another set of hardware with anything approaching accuracy. What would be awesome is if someone could figure out whether they'd need Raspberry Pi Zero to run some algorithm or if they could get away with just using a BBC Micro:Bit, by looking at a benchmark run submitted to the site from an i9-7940X. Sadly I suspect the amount of realtime introspection needed (memory speed, practical cache coherency (per level), bus saturation, instructions used per cycle) would make this difficult - particularly because, even if a given benchmark was going to go fishing for performance counter info (and I just learned that even the Gen1's ARM 1176-based PMU provides a few, including one that counts instructions), benchmarking memory I/O is a bit harder; PCI DMA MMIO debug cards only map a small range of memory, like 256k or so, and I don't know if such cards can back the region with actual RAM. I suspect the access latency would be so different than from normal RAM that even if this did exist it'd never be used. Sigh. And then differences in compiler optimization approach would have to be taken into account, and thorough understanding of assembly language for all target architecture(s) would be needed to have the time of day to page through the diff analysis... Hmm, I think "simple" got left behind a few thousand miles ago. It's kinda (morbidly?) fascinating how different that different architectures are on the ground, hah.

WalterBright 8y ago

Isaac Gouy is the gatekeeper for the shootout, and he refuses to include D for reasons he's refused to elucidate. I stopped publishing benchmarks myself 15 years ago as people always assumed my thumb was on the scale.

codyguy 8y ago
I was a bit surprised by the inclusion of lesser languages and the omission of D !! Please share the secret of your tenacity and inspiration in the face of such "setbacks". How do you do it on a day to day basis?
- WalterBright 8y ago
  I know how good D is and I have a lot of confidence in it. There are a lot of great people using it that are getting their jobs done faster & better and are thereby making more money. That's what matters to me. Everybody likes to make more money :-) Back when I was a brand new engineer at Boeing, I was talking to my lead engineer about the beauty of aerospace engineering. He smiled and said I didn't get it. Boeing wasn't making airplanes, they were building money making machines for airline companies. That the airplane turned out to be a beauty was just a happy side effect! D isn't about my personal aggrandizement. It's about how effective it is for users at solving their problems and making money for them.
e12e 8y ago
Has there been any recent discussion on this? I suspect earlier reasons might have included D not being packaged for Debian? It can be though to shake "early license impressions" - D (dmd specifically) is one example, Ada (FSF GNAT vs AdaCore's GNAT Pro and Ada-gnat - GPL w/o runtime exception) is another. [ed: if others are interested in "make your own measurements and host it yourself", relevant page with link to code etc appear to be: https://benchmarksgame-team.pages.debian.net/benchmarksgame/play.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... ]
- WalterBright 8y ago
  > any recent discussion Not since I asked him to stop dropping by the D forum to remind us that he wouldn't include D in the shootout. I've personally lost all interest in it. (The benchmarks are also small and too easily gamed by the author of the benchmark code, and the compiler developer. I encourage people to run timing tests on their own code, as that is what matters to them.)
  - girvo 8y ago
    My experience has been that I get better ideas of benchmarks by profiling my real application code, porting the hotspots to the new language or framework within a mostly bare skeleton and testing it. It’s not perfect, but gives me a better idea as to whether moving will actually help!
  - petre 8y ago
    I did run my own prime number crunching benchmark for fun and D blew Rust away but lagged behind Go. I used mutable Vec and HashSet in Rust, associative arrays and arrays in D, maps and arrays in Go to store the sieves.
    - dvfjsdhgfv 8y ago
      You see, that's the point - even if you do something as simple, there are many ways to do it (and different compiler versions, especially in the case of D), different optimizations at the code and compiler level and all that - it's practically impossible to have a reliable, comprehensive benchmark. (That said, D is lightning fast for me!)
    - steveklabnik 8y ago
      HashSet uses a cryptographically secure hashing algorithm by default, it’s gonna be slow.
      - petre 8y ago
        What's the Rust equivalent of Go maps or D associative arrays then?
igouy 8y ago
>> for reasons he's refused to elucidate For the simplest reasons that I have stated to you many times! Including every language implementation that the language author would like to have included is more work than I am willing to do, period. Sept 13 2008: 63 language implementations were shown- https://web.archive.org/web/20080913030117/http://shootout.alioth.debian.org:80/gp4/ https://web.archive.org/web/20080913030117/http://shootout.a... - currently, 27 language implementations are shown. You could truthfully say "he refuses to include [at-least 30 language implementations]". In that regard, there's nothing special about D.
- exikyut 8y ago
  I have no context regarding the history of the benchmarks game. With that in mind, I have a question: What about you create a technical specification (including both technical depth and common-sense breadth) for what kind of benchmark you'll accept, throw the whole thing on GitHub, and then refuse 99% of pull requests? :) (ie, only accept really really good quality benchmark implementations) Eventually, enough developers unimpressed that Language X is not adequately represented would step up to the plate and maintain good-quality benchmark code. (This could get pretty interesting with rapidly-evolving languages like Rust.) Obviously this is all very ideal and I can see so many ways such an endeavor could go horribly wrong, sure. I can also very easily see you having floated such an idea then discarded it for reasons I haven't even thought of.
  - igouy 8y ago
    What about I work to publish crowd-sourced programs and measurement scripts: so that anyone can make their own comparisons, on their own hardware, against whatever other language implementations they want to write programs for :-) https://benchmarksgame-team.pages.debian.net/benchmarksgame/play.html#languagex https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
    - exikyut 8y ago
      Ah; that certainly works too! The only thing I wonder about is allowing any given set of hardware to be compared with another set of hardware with anything approaching accuracy. What would be awesome is if someone could figure out whether they'd need Raspberry Pi Zero to run some algorithm or if they could get away with just using a BBC Micro:Bit, by looking at a benchmark run submitted to the site from an i9-7940X. Sadly I suspect the amount of realtime introspection needed (memory speed, practical cache coherency (per level), bus saturation, instructions used per cycle) would make this difficult - particularly because, even if a given benchmark was going to go fishing for performance counter info (and I just learned that even the Gen1's ARM 1176-based PMU provides a few, including one that counts instructions), benchmarking memory I/O is a bit harder; PCI DMA MMIO debug cards only map a small range of memory, like 256k or so, and I don't know if such cards can back the region with actual RAM. I suspect the access latency would be so different than from normal RAM that even if this did exist it'd never be used. Sigh. And then differences in compiler optimization approach would have to be taken into account, and thorough understanding of assembly language for all target architecture(s) would be needed to have the time of day to page through the diff analysis... Hmm, I think "simple" got left behind a few thousand miles ago. It's kinda (morbidly?) fascinating how different that different architectures are on the ground, hah.