3 ms·
I ran 10k test locally on 2e5 and I'm seeing 4 orders of magnitude instability, but very high local stability (i.e. runs within specific second are very stable,
by xlii 3mo ago
I ran 10k test locally on 2e5 and I'm seeing 4 orders of magnitude instability, but very high local stability (i.e. runs within specific second are very stable, showing almost no deviation, runs couple second later are the same, but results are within 1 OoM of the prior results (smaller batches, 500 tests).
I'm not saying that optimization isn't valid, what I'm saying is that Quicksort shouldn't be optimized over randomized per run data set.
- marginalia_nu 3mo agoYou don't happen to be running `srand(time(0))`? > I'm not saying that optimization isn't valid, what I'm saying is that Quicksort shouldn't be optimized over randomized per run data set. But yeah, this is correct. When optimizing, we want to pin every variable other than the change, as much as possible.
- atn34 3mo agoThen you run the risk of overfitting to whatever seed you picked. I think fixing a seed is probably a good idea most of the time but it’s worth considering