2 ms·
Without knowing the circumstances of how the bug was introduced, and the testing infrastructure, I'm torn. On the one hand, I could be convinced that some belie
by mikestew 9y ago
Without knowing the circumstances of how the bug was introduced, and the testing infrastructure, I'm torn. On the one hand, I could be convinced that some believable scenarios exist where test might have missed this through something other than incompetence and sloppiness.
OTOH, I'm having a hard time imagining what that "believable" scenario might look like. You hand me Disk Utility to test, and I'm going to do some exploratory testing to cram some large strings in both fields, yada yada, we all know the drill. You know, take a moment or two before hitting the code editor. I'm just grasping to figure out how I wouldn't find this bug in like five minutes. And that's before formal tests start getting written. There's missing a bug, and then there's "maybe testing software isn't for you".
In summary, I very much want to give Apple the benefit of the doubt here, but I'm sure having a hard time of it.
- Someone 9y agoI think that bug could easily slip through once, even with a test script. Let’s say the script says - enter P as password - enter H as hint - click “OK” - unmount the disk - mount it again - check that the dialog shows the hint H - enter P as the password ... Now, if P isn’t obviously a password and H isn’t obviously a hint (let’s say P is ‘foo’, and H is ‘bar’), it takes only one slip of the mind to turn that “check that the dialog shows the hint H” into “check that the dialog shows the string you just entered as hint”, and from there, it’s only a tiny further slip to “yes, that’s a string I just entered”. To me, it seems at least as likely that that gets past a tester once as it is that a programmer writes that bug. It still should be highly unlikely to get throug multiple rounds of testing by multiple testers, though. A better test description would use a more password-like value for P (say ‘tqbfjoald’) and a more hint-like value for H (‘quick and brown’), decreasing that risk.
- mikestew 9y agoA better test description would use a more password-like value for P (say ‘tqbfjoald’) and a more hint-like value for H (‘quick and brown’), decreasing that risk. Maybe I expect too much, but a tester with any experience is going to use strings like "thePassword" and "theHint" to reduce those brain farts. One doesn't have to test software very long before discovering why using "test", "test" as the respective strings for that dialog will bite you. I agree with your hypothesis, but it's one of those mistakes I'd expect out of a fresh-out-of-college person, and I would expect those folks to be testing, say, TextEdit and not security-sensitive pieces. But there are so many unknowns that I still reserve judgement. I'd just like to know what piece of the process broke down such that something like this gets out the door.
- Someone 9y agoI agree it shouldn’t happen, but in this case, the programmer presumably made (more or less) the same mistake, writing out a variable called password unencrypted, possibly after copying it to a variable or structure field called passwordhint. Thinking of that, this may be an example of a copy-paste bug (https://www.viva64.com/en/a/0068/ https://www.viva64.com/en/a/0068/), where the programmer wrote strcpy(&r.password , &encryptedpassword); strcpy(&r.passwordhint, &password); instead of strcpy(&r.password , &encryptedpassword); strcpy(&r.passwordhint, &passwordhint); (Hopefully using the safer strncpy, but that’s orthogonal to this argument)