4 ms·

What is the value in using a system that you admit is in beta and not optimized to benchmark two different hardware platforms?

supergreg 11y ago
To see how it changes as you work on it?
geerlingguy 11y ago
I've been benchmarking Drupal 8 since early alphas (not all benchmarks are published on that Wiki page... I really need to update it), as I'm interested in seeing how different infrastructure, different PHP versions, and different caching architectures can help or hurt performance. See, for instance: https://twitter.com/geerlingguy/status/702909750405517312 https://twitter.com/geerlingguy/status/702909750405517312 Drupal 8.0.4 is the current release, and I'm going to be rerunning all benchmarks on that version soon.
Ensorceled 11y ago
To call attention to the fact that it needs that optimization?
- radicalbyte 11y ago
  He's benchmarking the difference between a Pi2 and a server, so it makes no sense to introduce another variable - unoptimized, unfinished code.
  - geerlingguy 11y ago
    As long as the code itself is the same... It's not a variable. I always benchmark against the same point release or git hash. It's nice to have comparisons of e.g. Beta 1 vs Beta 2, so I can find out where performance regressions are introduced into the code.
    - radicalbyte 11y ago
      The problem is the increased risk of nonsense benchmarks. I.e. if the unoptimized code is super slow because the naive implementation compiles to something unsuited to, say, the x86 microarchitecture. Or that performance is limited by lock contention. That's way more likely with pre-release then production-grate software. If you want to compare B1 and B2 that's great - but then you post to hackernews a comparison between two Drupal betas, not a comparison between two hardware platforms.
    - radicalbyte 11y ago
      Benchmarking beta software is of course great and very valuable - it focuses attention and will hopefully motivate projects to improve performance. It's just strange to do it in a piece which is predominantly arguing for the effectiveness of a cheap hardware solution :)
TickleSteve 11y ago
Because he's benchmarking the system, not drupal....

supergreg 11y ago

To see how it changes as you work on it?

geerlingguy 11y ago

I've been benchmarking Drupal 8 since early alphas (not all benchmarks are published on that Wiki page... I really need to update it), as I'm interested in seeing how different infrastructure, different PHP versions, and different caching architectures can help or hurt performance. See, for instance: https://twitter.com/geerlingguy/status/702909750405517312 https://twitter.com/geerlingguy/status/702909750405517312 Drupal 8.0.4 is the current release, and I'm going to be rerunning all benchmarks on that version soon.

Ensorceled 11y ago

To call attention to the fact that it needs that optimization?

radicalbyte 11y ago
He's benchmarking the difference between a Pi2 and a server, so it makes no sense to introduce another variable - unoptimized, unfinished code.
- geerlingguy 11y ago
  As long as the code itself is the same... It's not a variable. I always benchmark against the same point release or git hash. It's nice to have comparisons of e.g. Beta 1 vs Beta 2, so I can find out where performance regressions are introduced into the code.
  - radicalbyte 11y ago
    The problem is the increased risk of nonsense benchmarks. I.e. if the unoptimized code is super slow because the naive implementation compiles to something unsuited to, say, the x86 microarchitecture. Or that performance is limited by lock contention. That's way more likely with pre-release then production-grate software. If you want to compare B1 and B2 that's great - but then you post to hackernews a comparison between two Drupal betas, not a comparison between two hardware platforms.
  - radicalbyte 11y ago
    Benchmarking beta software is of course great and very valuable - it focuses attention and will hopefully motivate projects to improve performance. It's just strange to do it in a piece which is predominantly arguing for the effectiveness of a cheap hardware solution :)

TickleSteve 11y ago

Because he's benchmarking the system, not drupal....