3 ms·
>he is testing one aspect in isolation of others. This is a good thing, and is generally the way in which scientific tests advance knowledge. Yes, maintaining
by bxr 15y ago
>he is testing one aspect in isolation of others. This is a good thing, and is generally the way in which scientific tests advance knowledge.
Yes, maintaining constants and only altering one variable. But this test has an 2nd variable, that is altered into a special state for all of the experimentation, and then we make the leap into assuming that these results are anything other than pretty to look at when the 2nd variable is in any other position.
This is a good test for seeing which server runs best with a load testing client competing for resources on the box and using the loopback network. Thats it.
The problem is you can't test without a network stack at all, you can only test with a different network stack. Whats to say the particulars of the loopback network aren't causing more issues to be introduced to the validity of the results than the full TCP/IP stack you're trying to avoid influencing the results? Nothing, that's what.
- palish 15y agoI'm happy people have begun to see that scientific tests generally have massive assumptions hidden deep within them, or quietly brushed underneath them. Personally, I find this to be a welcome change from the typical "oooh-look-at-the-shiny-graphs" mentality that has become so pervasive on HN with regards to performance testing. Keep up this spirit of questioning, and you'll discover a lot of interesting secrets. (For example, a placebo is 87% as effective as the 5 major antidepressant brands. Also, the CIE 1931 "color space" is a flawed system which is only roughly accurate.) In general, anyone who presents statistics (of anything) should be subjected to a lot of skepticism. Scientific knowledge can only advance by truly testing one variable and only one variable.