4 ms·

The problem you're having is you can't figure out how to use benchmarks to understand how a system works. You're throwing up your hands and saying, "TOO DIFFER

The problem you're having is you can't figure out how to use benchmarks to understand how a system works. You're throwing up your hands and saying, "TOO DIFFERENT FROM REALITY!" Other folks don't do that, and while not 1:1, they are able to correlate the performance of a benchmark with the performance of their own use case. Try not to get wrapped around the axle on "everyone", by the way, it's not literal.

peterwwillis 7y ago
> Other folks don't do that, and while not 1:1, they are able to correlate the performance of a benchmark with the performance of their own use case. You can correlate lots of things. Poor countries increase penis size. Ice cream leads to murder. Cheese kills people by tangling them in their bedsheets. These things are strongly correlated. But they have no causative relationship. To put it another way, a single benchmark test used find a solution is scientifically equivalent to finding one "average" man, making him run a series of athletic tests, and making his performance the benchmark for men's athletics.
- diminoten 7y ago
  Sure, if you do it wrong. But if you do it right, you can get a lot of informative, valuable data out of a benchmark result that can accurately help decide between two (or more) technologies. Like I've been saying, benchmark results are plenty useful if you know how to use them.
dragonwriter 7y ago
> The problem you're having Again, I'm not having a problem. > is you can't figure out how to use benchmarks to understand how a system works No, I have no partucular problem evaluating whether a benchmark is useful to a decision and if so, how. Nor have I said anything indicating any such problem. > You're throwing up your hands and saying, "TOO DIFFERENT FROM REALITY!" No, I'm saying he naive statement upthread that benchmark performance correlates with every possible use of a technology is nonsense. That's it. > Other folks don't do that Actually, some do, but that's neither here nor there. > and while not 1:1, they are able to correlate the performance of a benchmark with the performance of their own use case. Once again, yes, lots of people have the skill to figure out whether and in what way particular benchmarks have utility for their usecases. I explicitly said that in the post you are responding to. That's very different than your claim that I reacted against, which is that any benchmark inherently correlates with every use case, which is—again—complete nonsense.
- diminoten 7y ago
  > any benchmark inherently correlates with every use case Oh. That's what you thought I said? Right. I didn't mean to say that, and if somehow I did say that I withdraw. Benchmarks are valuable if you know how to use them, but of course I agree with you, a benchmark isn't always relevant to every use case. Side note, quoting many small portions of a person's comment and replying exclusively to the quoted bits is inferior to replying to them with full sentences/paragraphs. I've only ever seen what you're doing done by folks who are super interested in arguing and completely uninterested in having a conversation. It definitely comes across like you do have a problem, which you have repeatedly stated you do not (and I believe you)! It just muddies the water, what you're doing here.

peterwwillis 7y ago

> Other folks don't do that, and while not 1:1, they are able to correlate the performance of a benchmark with the performance of their own use case. You can correlate lots of things. Poor countries increase penis size. Ice cream leads to murder. Cheese kills people by tangling them in their bedsheets. These things are strongly correlated. But they have no causative relationship. To put it another way, a single benchmark test used find a solution is scientifically equivalent to finding one "average" man, making him run a series of athletic tests, and making his performance the benchmark for men's athletics.

diminoten 7y ago
Sure, if you do it wrong. But if you do it right, you can get a lot of informative, valuable data out of a benchmark result that can accurately help decide between two (or more) technologies. Like I've been saying, benchmark results are plenty useful if you know how to use them.

dragonwriter 7y ago

> The problem you're having Again, I'm not having a problem. > is you can't figure out how to use benchmarks to understand how a system works No, I have no partucular problem evaluating whether a benchmark is useful to a decision and if so, how. Nor have I said anything indicating any such problem. > You're throwing up your hands and saying, "TOO DIFFERENT FROM REALITY!" No, I'm saying he naive statement upthread that benchmark performance correlates with every possible use of a technology is nonsense. That's it. > Other folks don't do that Actually, some do, but that's neither here nor there. > and while not 1:1, they are able to correlate the performance of a benchmark with the performance of their own use case. Once again, yes, lots of people have the skill to figure out whether and in what way particular benchmarks have utility for their usecases. I explicitly said that in the post you are responding to. That's very different than your claim that I reacted against, which is that any benchmark inherently correlates with every use case, which is—again—complete nonsense.

diminoten 7y ago
> any benchmark inherently correlates with every use case Oh. That's what you thought I said? Right. I didn't mean to say that, and if somehow I did say that I withdraw. Benchmarks are valuable if you know how to use them, but of course I agree with you, a benchmark isn't always relevant to every use case. Side note, quoting many small portions of a person's comment and replying exclusively to the quoted bits is inferior to replying to them with full sentences/paragraphs. I've only ever seen what you're doing done by folks who are super interested in arguing and completely uninterested in having a conversation. It definitely comes across like you do have a problem, which you have repeatedly stated you do not (and I believe you)! It just muddies the water, what you're doing here.