4 ms·

> You see there are 52,363 commits between those two kernels, Would it have been possible to generate a list of commits on timeline and perform bisects in a me

by amtamt 3y ago

> You see there are 52,363 commits between those two kernels, Would it have been possible to generate a list of commits on timeline and perform bisects in a method similar to binary search? Or that was already done and still 293k boots were needed? I am genuinely curious.

detaro 3y ago
> in a method similar to binary search? That's what bisect means. You do a binary search for the breaking commit
- kangalioo 3y ago
  Exactly, and the reason 293k boots were needed anyways is because the bug was hard to reproduce and needed thousands of boots to potentially trigger
  - hinkley 3y ago
    That’s why I always push telemetry onto a team. Concurrency and resource bugs are extremely expensive per iteration. Anything that can hint that the problem you are certain happened somewhere in a two month period was more likely to have happened in a particular week saves you a ton of work. The human brain struggles when it has to execute almost, but not quite, the same exercises over and over again. It becomes a blur and after a while you can’t recall if you ran all of the steps this cycle (if you can’t automate the test well) and all it takes is typing “bad” when you meant “good” to end in failure. Which doesn’t sound like a big likelihood but when you’re testing a negative it’s much easier to get your wires crossed.
cwillu 3y ago
Expanding on other comments: binary search is extremely sensitive to false positives and false negatives… false information generally. While other approaches will still put you in the right ballpark even with some amount of incorrect information, a binary search with a single flipped bit will consistently land you in entirely unrelated areas, so you need to be _damn_ sure your “git bisect good/bad” is correct.
- rwmj 3y ago
  In the other post there was an interesting discussion about different approaches to searching for a bug (other than straight bisection) when your test is unreliable: https://news.ycombinator.com/item?id=36327263 https://news.ycombinator.com/item?id=36327263
- wging 3y ago
  There is `git bisect skip` (https://git-scm.com/docs/git-bisect#_bisect_skip https://git-scm.com/docs/git-bisect#_bisect_skip), which lets you cope with a testing process that sometimes doesn't yield a true/false result. But it's more useful for untestable commits than for inconclusive results like you'd get from a single boot here.

by amtamt 3y ago

detaro 3y ago

> in a method similar to binary search? That's what bisect means. You do a binary search for the breaking commit

kangalioo 3y ago
Exactly, and the reason 293k boots were needed anyways is because the bug was hard to reproduce and needed thousands of boots to potentially trigger
- hinkley 3y ago
  That’s why I always push telemetry onto a team. Concurrency and resource bugs are extremely expensive per iteration. Anything that can hint that the problem you are certain happened somewhere in a two month period was more likely to have happened in a particular week saves you a ton of work. The human brain struggles when it has to execute almost, but not quite, the same exercises over and over again. It becomes a blur and after a while you can’t recall if you ran all of the steps this cycle (if you can’t automate the test well) and all it takes is typing “bad” when you meant “good” to end in failure. Which doesn’t sound like a big likelihood but when you’re testing a negative it’s much easier to get your wires crossed.

cwillu 3y ago

Expanding on other comments: binary search is extremely sensitive to false positives and false negatives… false information generally. While other approaches will still put you in the right ballpark even with some amount of incorrect information, a binary search with a single flipped bit will consistently land you in entirely unrelated areas, so you need to be _damn_ sure your “git bisect good/bad” is correct.

rwmj 3y ago
In the other post there was an interesting discussion about different approaches to searching for a bug (other than straight bisection) when your test is unreliable: https://news.ycombinator.com/item?id=36327263 https://news.ycombinator.com/item?id=36327263
wging 3y ago
There is `git bisect skip` (https://git-scm.com/docs/git-bisect#_bisect_skip https://git-scm.com/docs/git-bisect#_bisect_skip), which lets you cope with a testing process that sometimes doesn't yield a true/false result. But it's more useful for untestable commits than for inconclusive results like you'd get from a single boot here.