5 ms·

> But for a hardware test, I'd expect a single running app, executing shared native codebase, in no disruptions airplane mode, measuring specific part of the ha

> But for a hardware test, I'd expect a single running app, executing shared native codebase, in no disruptions airplane mode, measuring specific part of the hardware. And that's not even close to measuring before/after improved branch prediction, which could be useful in both phones. I would argue that would be a completely useless test unless you're explicitly testing something for development purposes only. Users use phones. Nothing else matters but the user experience. If you can make your phone beat another phone in your limited, no disruptions, network traffic off example that doesn't mean anything. I feel this is a fair test. You test the phones at what they're supposed to be able to do. If one does it better it's legit to point that out regardless of what the underlying structure looks like.

viraptor 10y ago
I think you either misread my comment or missed the point of it. Overall tests matter, specific tests matter, unittests matter, microbenchmarks matter, user experience matters, perceived performance matters. But they don't always intersect. Bringing up user experience when talking about branch predictors is meaningless. Just as bringing up branch predictors when talking about perceived performance. (unless you're proving that this exact feature being available/not available, with other things being controlled for, makes a major difference in the test)
- zmb_ 10y ago
  > Bringing up user experience when talking about branch predictors is meaningless. If working on branch predictors does not impact the user experience, then arguably that work is meaningless in the context of building real products -- it may still be interesting for academic purposes.
  - fasouto 10y ago
    Branch predictors made the phones faster and this affects the user experience. But if you want to test the performance of branch predictors you have to compare similar things, you cannot benchmark two different OS and apps because you get and OS and app test not an processor test.
    - woliveirajr 10y ago
      I think the point that previous posters were making is that in controlled environments some features might outperform others. And yes, the scientific way of testing things is isolating variables and changing one thing each time. But, for the final user, it doesn't matter if each technology alone is better than another. If you have a background 64-core processor, driven by another 8-core... What matters is how fast something loads, performs, and how long batteries last. Of course, this might be outplaced in a thread discussing NN branch prediction.

viraptor 10y ago

I think you either misread my comment or missed the point of it. Overall tests matter, specific tests matter, unittests matter, microbenchmarks matter, user experience matters, perceived performance matters. But they don't always intersect. Bringing up user experience when talking about branch predictors is meaningless. Just as bringing up branch predictors when talking about perceived performance. (unless you're proving that this exact feature being available/not available, with other things being controlled for, makes a major difference in the test)

zmb_ 10y ago
> Bringing up user experience when talking about branch predictors is meaningless. If working on branch predictors does not impact the user experience, then arguably that work is meaningless in the context of building real products -- it may still be interesting for academic purposes.
- fasouto 10y ago
  Branch predictors made the phones faster and this affects the user experience. But if you want to test the performance of branch predictors you have to compare similar things, you cannot benchmark two different OS and apps because you get and OS and app test not an processor test.
  - woliveirajr 10y ago
    I think the point that previous posters were making is that in controlled environments some features might outperform others. And yes, the scientific way of testing things is isolating variables and changing one thing each time. But, for the final user, it doesn't matter if each technology alone is better than another. If you have a background 64-core processor, driven by another 8-core... What matters is how fast something loads, performs, and how long batteries last. Of course, this might be outplaced in a thread discussing NN branch prediction.