5 ms·

How can such a benchmark be in any way informative? How much am I allowed to optimize the nim code? How much am I allowed to optimize the Rust code? Am I allowe

How can such a benchmark be in any way informative? How much am I allowed to optimize the nim code? How much am I allowed to optimize the Rust code? Am I allowed to just change nim's GC to ARC/ORC? I'm sorry but this just does not make any sense to me.

onebot 5y ago
It is more informative than nothing. It shows different languages implementing the same functions and the performance of those. It is something to compare to. Maybe they can all be optimized, etc. But you need to have some kind of starting point. I think people tend to poo-poo benchmarks (unless their fav language at top) and claim they don't represent real world. But I always argue that they at the very minimum give you some context for comparison. If you look at how they are implemented you can get further context. I only wish the benchmarks had some metric for how idiomatic the implementation is compared to normal developers.
- igouy 5y ago
  Yes. https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#starting-point https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html#non-motivation https://benchmarksgame-team.pages.debian.net/benchmarksgame/... ---- > I only wish the benchmarks had some metric for how idiomatic the implementation is compared to normal developers. Maybe the benchmarks game website kind-of does — source code size. Otherwise "I only wish…" so — wishful thinking. Otherwise "how idiomatic" seems to mean — how I would implement the program today, if I didn't care [goldilocks] too much about performance? (Incidentally "normal developers" doesn't seem to be stretching towards inclusive.)
gameswithgo 5y ago
All benchmarks are informative, also all benchmarks are imperfect and incomplete, for reasons you have specified and many others. However it is a mistake to then suppose that nothing can be known and just go ahead and use Python to make a AAA first person shooter.
- Mikeb85 5y ago
  On the flipside slow languages (Python included) are used to script some AAA games. And no language other than C++ has really found traction for stuff like engines; not even Rust...
  - loeg 5y ago
    What AAA games use Python?
    - Verdex 5y ago
      I think Eve Online uses python.
      - gameswithgo 5y ago
        They do, and its caused them a lot of headache, probably was a mistake. If they would have at least used something like C#/Java/Go (go didn't exist at the time) they might not have had to do time-slowdowns to make large battles possible. Then again maybe it would have slowed down development and never gotten off the ground at all!
      - dragonwriter 5y ago
        IIRC, Eve used Stackless Python, not normal CPython.
    - planetis 5y ago
      Civ6 btw
      - ryl00 5y ago
        I thought only Civ IV used python.

onebot 5y ago

It is more informative than nothing. It shows different languages implementing the same functions and the performance of those. It is something to compare to. Maybe they can all be optimized, etc. But you need to have some kind of starting point. I think people tend to poo-poo benchmarks (unless their fav language at top) and claim they don't represent real world. But I always argue that they at the very minimum give you some context for comparison. If you look at how they are implemented you can get further context. I only wish the benchmarks had some metric for how idiomatic the implementation is compared to normal developers.

igouy 5y ago
Yes. https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#starting-point https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html#non-motivation https://benchmarksgame-team.pages.debian.net/benchmarksgame/... ---- > I only wish the benchmarks had some metric for how idiomatic the implementation is compared to normal developers. Maybe the benchmarks game website kind-of does — source code size. Otherwise "I only wish…" so — wishful thinking. Otherwise "how idiomatic" seems to mean — how I would implement the program today, if I didn't care [goldilocks] too much about performance? (Incidentally "normal developers" doesn't seem to be stretching towards inclusive.)

gameswithgo 5y ago

All benchmarks are informative, also all benchmarks are imperfect and incomplete, for reasons you have specified and many others. However it is a mistake to then suppose that nothing can be known and just go ahead and use Python to make a AAA first person shooter.

Mikeb85 5y ago
On the flipside slow languages (Python included) are used to script some AAA games. And no language other than C++ has really found traction for stuff like engines; not even Rust...
- loeg 5y ago
  What AAA games use Python?
  - Verdex 5y ago
    I think Eve Online uses python.
    - gameswithgo 5y ago
      They do, and its caused them a lot of headache, probably was a mistake. If they would have at least used something like C#/Java/Go (go didn't exist at the time) they might not have had to do time-slowdowns to make large battles possible. Then again maybe it would have slowed down development and never gotten off the ground at all!
    - dragonwriter 5y ago
      IIRC, Eve used Stackless Python, not normal CPython.
  - planetis 5y ago
    Civ6 btw
    - ryl00 5y ago
      I thought only Civ IV used python.