5 ms·

Another point of clarification that is of great importance to the results, and is a common Rust newcomer error: The benchmarks for the Rust implementation (in t

Another point of clarification that is of great importance to the results, and is a common Rust newcomer error: The benchmarks for the Rust implementation (in the original post that got all the traction) were run with a /debug/ build of rust, i.e. not an optimized binary compiled with --release. So it was comparing something that a) didn't do meaningful parsing against b) the full parsing rust implementation in a non-optimized debug build.

SushiHippie 3y ago
Am I missing something? In the git repository [0] it says: > needletail_benchmark folder was compiled using the command cargo build --release and ran using the following command ./target/release/<binary> <path/to/file.fq>. Or are you talking about something else here? [0] https://github.com/MoSafi2/MojoFastTrim https://github.com/MoSafi2/MojoFastTrim
- mcqueenjordan 3y ago
  It was later edited, after it had basically made the rounds.
  - SushiHippie 3y ago
    Ah okay, found the commit that changed the benchmark numbers https://github.com/MoSafi2/MojoFastTrim/commit/530bffaf21663d764c4f503de967ae0420102d45 https://github.com/MoSafi2/MojoFastTrim/commit/530bffaf21663...
bhansconnect 3y ago
This is not accurate. The blog post used `--release` for it's Rust numbers. The confusion comes from the 50% performance win being specific to running on an M2 mac. On an x86_64 Linux machine, the results are more or less equivalent.
tehsauce 3y ago
How much does this particular result change when running in release mode?
- alpaca128 3y ago
  Depending on the code I've seen performance increases above 100x in some cases. While that's not exactly the norm, benchmarking Rust in debug mode is absolutely pointless even as a rough estimate.
  - eyegor 3y ago
    Is there any compiled language that doesn't benefit heavily from release builds? That would be interesting if true.
    - adgjlsfhk1 3y ago
      Julia, for example runs by default with -O2 and debug info turned on. It's a good combo between debug-ability and performance.
    - pornel 3y ago
      This can happen in languages that use dynamic constructs that can't be optimized out. For example, there was a PHP-to-native compiler (HipHop/HPHPc) that lost to faster interpreters and JIT. Apple's Rosetta 2 translates x86-64 to aarch64 that runs surprisingly fast, despite being mostly a straightforward translation of instructions, rather than something clever like a recompiling optimizing JIT. And the plain old C is relatively fast without optimizations, because it doesn't rely on abstraction layers being optimized out.
- fwip 3y ago
  On my machine, running the debug executable on the medium-size dataset takes ~14.5 seconds, and release mode takes ~0.8 seconds.
  - adgjlsfhk1 3y ago

SushiHippie 3y ago

Am I missing something? In the git repository [0] it says: > needletail_benchmark folder was compiled using the command cargo build --release and ran using the following command ./target/release/<binary> <path/to/file.fq>. Or are you talking about something else here? [0] https://github.com/MoSafi2/MojoFastTrim https://github.com/MoSafi2/MojoFastTrim

mcqueenjordan 3y ago
It was later edited, after it had basically made the rounds.
- SushiHippie 3y ago
  Ah okay, found the commit that changed the benchmark numbers https://github.com/MoSafi2/MojoFastTrim/commit/530bffaf21663d764c4f503de967ae0420102d45 https://github.com/MoSafi2/MojoFastTrim/commit/530bffaf21663...

bhansconnect 3y ago

This is not accurate. The blog post used `--release` for it's Rust numbers. The confusion comes from the 50% performance win being specific to running on an M2 mac. On an x86_64 Linux machine, the results are more or less equivalent.

tehsauce 3y ago

How much does this particular result change when running in release mode?

alpaca128 3y ago
Depending on the code I've seen performance increases above 100x in some cases. While that's not exactly the norm, benchmarking Rust in debug mode is absolutely pointless even as a rough estimate.
- eyegor 3y ago
  Is there any compiled language that doesn't benefit heavily from release builds? That would be interesting if true.
  - adgjlsfhk1 3y ago
    Julia, for example runs by default with -O2 and debug info turned on. It's a good combo between debug-ability and performance.
  - pornel 3y ago
    This can happen in languages that use dynamic constructs that can't be optimized out. For example, there was a PHP-to-native compiler (HipHop/HPHPc) that lost to faster interpreters and JIT. Apple's Rosetta 2 translates x86-64 to aarch64 that runs surprisingly fast, despite being mostly a straightforward translation of instructions, rather than something clever like a recompiling optimizing JIT. And the plain old C is relatively fast without optimizations, because it doesn't rely on abstraction layers being optimized out.
fwip 3y ago
On my machine, running the debug executable on the medium-size dataset takes ~14.5 seconds, and release mode takes ~0.8 seconds.
- adgjlsfhk1 3y ago