3 ms·

"If you wrote buggy software, why would the software you wrote to check that software be any less buggy?" I work in chip design, where bugs can be rather costl

"If you wrote buggy software, why would the software you wrote to check that software be any less buggy?" I work in chip design, where bugs can be rather costly, e.g. $1mm for a new mask set, not to mention the months it takes to get back new hw. The situation described above is why we try to have one person do the design and a different person do the verification/testing. A lot of the time, the test writer will treat the design as a black box, not even look at the code, and instead verify to an external specification. (There's also white box testing, where you try to target specifc areas of the design that might be especially problematic.) This does protect against issues like misinterpreted requirements, invalid assumptions of valid input and/or operating scenarios. But I think it also ends up as a lot more work, to have two people get familiar with whatever it is you're designing. The best method is probably a combination of both.

ereyes01 8y ago
When you have a well thought-out external specification, it really gives you lots of good material from which to derive all the right tests while still treating your implementation as a black box. Unless you are implementing an established RFC or some other well-defined system, such a tight specification rarely exists in a software setting, and is very difficult to effectively design a priori. For this reason, techniques like test-driven development have arisen to try to make the programmer repeatedly and consistently assert their understanding of the specification via unit tests, while evolving those assumptions as needs and understandings change.
spc476 8y ago
When I was doing QA at work, I never used the official tools to generate the data files required for testing the application, but wrote my own. This brought to light a ton of issues and unwritten assumptions (both in my code, and in the application).

ereyes01 8y ago

When you have a well thought-out external specification, it really gives you lots of good material from which to derive all the right tests while still treating your implementation as a black box. Unless you are implementing an established RFC or some other well-defined system, such a tight specification rarely exists in a software setting, and is very difficult to effectively design a priori. For this reason, techniques like test-driven development have arisen to try to make the programmer repeatedly and consistently assert their understanding of the specification via unit tests, while evolving those assumptions as needs and understandings change.

spc476 8y ago

When I was doing QA at work, I never used the official tools to generate the data files required for testing the application, but wrote my own. This brought to light a ton of issues and unwritten assumptions (both in my code, and in the application).