3 ms·

Reading the future ideas section: * Async support not there yet * statistical power is principled in terms of following a good paper about how to pick how man

Reading the future ideas section: * Async support not there yet * statistical power is principled in terms of following a good paper about how to pick how many iterations, but the variance stuff isn’t implemented yet. By definition, if I’m reading the readme correctly, its execution mode is equivalent to —quick. * baseline evaluation not there yet. My overall gripe remains that this stuff isn’t available natively within the Rust/cargo ecosystem. I should be able to swap out benchmarking frameworks without having to rewrite my entire codebase to try out a new one.

jeffparsons 3y ago
The recently-formed testing subteam is looking to solve your exact complaint: https://github.com/rust-lang/rfcs/pull/3455 https://github.com/rust-lang/rfcs/pull/3455
- vlovich123 3y ago
  That's positive news, but I'll have to reserve my enthusiasm. I've been burned too many times with what seem like reasonable-looking RFCs that seem to have some initial momentum and then get closed out after a few years with no progress being made (& how that's communicated out & tracked is another complaint). For example & relevant to this: https://github.com/rust-lang/rfcs/issues/816 https://github.com/rust-lang/rfcs/issues/816 redirected to https://github.com/rust-lang/rfcs/pull/2318 https://github.com/rust-lang/rfcs/pull/2318 https://github.com/rust-lang/rfcs/pull/2318 https://github.com/rust-lang/rfcs/pull/2318 says merged but links to tracking issue https://github.com/rust-lang/rust/issues/50297 https://github.com/rust-lang/rust/issues/50297 which was closed out without ever being implemented. So 5 years with no progress & a restarted effort. Don't get me wrong. I appreciate all the people devoting energy to this & other issues, & it's certainly easy to provide criticism from the sidelines, especially since actually solving the problem I'm sure can be hard & I'm not putting my cycles toward this. All I'm trying to communicate is that I'm personally not going to get my hopes up but I really wish the team luck and hope progress can be made here finally!

jeffparsons 3y ago

The recently-formed testing subteam is looking to solve your exact complaint: https://github.com/rust-lang/rfcs/pull/3455 https://github.com/rust-lang/rfcs/pull/3455

vlovich123 3y ago
That's positive news, but I'll have to reserve my enthusiasm. I've been burned too many times with what seem like reasonable-looking RFCs that seem to have some initial momentum and then get closed out after a few years with no progress being made (& how that's communicated out & tracked is another complaint). For example & relevant to this: https://github.com/rust-lang/rfcs/issues/816 https://github.com/rust-lang/rfcs/issues/816 redirected to https://github.com/rust-lang/rfcs/pull/2318 https://github.com/rust-lang/rfcs/pull/2318 https://github.com/rust-lang/rfcs/pull/2318 https://github.com/rust-lang/rfcs/pull/2318 says merged but links to tracking issue https://github.com/rust-lang/rust/issues/50297 https://github.com/rust-lang/rust/issues/50297 which was closed out without ever being implemented. So 5 years with no progress & a restarted effort. Don't get me wrong. I appreciate all the people devoting energy to this & other issues, & it's certainly easy to provide criticism from the sidelines, especially since actually solving the problem I'm sure can be hard & I'm not putting my cycles toward this. All I'm trying to communicate is that I'm personally not going to get my hopes up but I really wish the team luck and hope progress can be made here finally!