3 ms·

I wrote the blog entry for this round [1], and I would agree with ksec (elsewhere in these comments) in recommending that readers start there since it includes

by bhauer 8y ago

I wrote the blog entry for this round [1], and I would agree with ksec (elsewhere in these comments) in recommending that readers start there since it includes the highlights of this round. I apologize ahead of time for my humor. It's still a bit early out here in California, but if anyone has any questions, I will try to answer as I can! [1] https://www.techempower.com/blog/2018/06/06/framework-benchmarks-round-16/ https://www.techempower.com/blog/2018/06/06/framework-benchm...

tomaha 8y ago
Do you have any plans to add memory usage to the benchmarks? It adds an extra dimension and sometimes even shows if there's a problem with a specific implementation (e.g. in the benchmark game it's often a hint for a program that can be optimized if the ratio compared to other languages is off).
- bhauer 8y ago
  Yes! In fact, I just responded to a similar question over at the Rust Subreddit thread about the same topic. We do in fact capture dstat data while executing the tests but as of today do not render these in any way. You can find raw CSV output from dstat at our logs server. For example at [1] are stats for Grizzly while measuring the "json" test type. I will create an issue at the project's GitHub repo to begin a conversation about which stats to render. I'd like to select a meaningful bite-size set of stats that we can render into a table view. Basically another tab in the results view similar to the latency tab. [1] http://tfb-logs.techempower.com/round-16/final/grizzly/json/stats.txt http://tfb-logs.techempower.com/round-16/final/grizzly/json/...
  - tomaha 8y ago
    That's great. Thanks.
  - drunner 8y ago
    If I could make a recommendation that would humor me personally, you could publicize the stats required to run the instance on an B1S or Fv2 Azure instance. I think it would be really interesting to see which frameworks are so costly upfront that they would impede a developer from prototyping with them on the cheap.
    - bhauer 8y ago
      I like that idea. I'll see what we can do!

by bhauer 8y ago

tomaha 8y ago

Do you have any plans to add memory usage to the benchmarks? It adds an extra dimension and sometimes even shows if there's a problem with a specific implementation (e.g. in the benchmark game it's often a hint for a program that can be optimized if the ratio compared to other languages is off).

bhauer 8y ago
Yes! In fact, I just responded to a similar question over at the Rust Subreddit thread about the same topic. We do in fact capture dstat data while executing the tests but as of today do not render these in any way. You can find raw CSV output from dstat at our logs server. For example at [1] are stats for Grizzly while measuring the "json" test type. I will create an issue at the project's GitHub repo to begin a conversation about which stats to render. I'd like to select a meaningful bite-size set of stats that we can render into a table view. Basically another tab in the results view similar to the latency tab. [1] http://tfb-logs.techempower.com/round-16/final/grizzly/json/stats.txt http://tfb-logs.techempower.com/round-16/final/grizzly/json/...
- tomaha 8y ago
  That's great. Thanks.
- drunner 8y ago
  If I could make a recommendation that would humor me personally, you could publicize the stats required to run the instance on an B1S or Fv2 Azure instance. I think it would be really interesting to see which frameworks are so costly upfront that they would impede a developer from prototyping with them on the cheap.
  - bhauer 8y ago
    I like that idea. I'll see what we can do!