4 ms·
I'm particularly interested in the idea of distribution/statistical-test driven software testing. Instead of concrete assertions (truth assertions, equality as
by jbr 17y ago
I'm particularly interested in the idea of distribution/statistical-test driven software testing. Instead of concrete assertions (truth assertions, equality assertions), we would make assertions about descriptive statistics or other tests. Or just assert that if we do something stochastic N times, we should expect some truth/equality test to pass at least M times.
I'm not sure what to do with that idea, but that's actually the core of why I posted this. Any other hackers out there excited by this idea?
- tristan_juricek 17y agoYes; I'm a lot weaker in my statistical understanding, but it seems like it would be a fantastic way of slicing up the testing pie. Unit tests are kind of handy if you've got some sort of "fencepost" style error where you need to figure out a piece of logic that was overlooked. But that's about it. This sort of testing would way more relevant, because it's a way of focusing on "what (data) do we 'well' right now"? I'm thinking that such sort of systems might be useful as a basis for performance or concurrency testing for example.
- pgbovine 17y agosome research projects that might be relevant (although not quite what you were thinking about): statistical bug isolation: http://pages.cs.wisc.edu/~liblit/pldi-2005/ http://pages.cs.wisc.edu/~liblit/pldi-2005/ CHESS - automated finding of concurrency bugs: http://research.microsoft.com/en-us/projects/chess/ http://research.microsoft.com/en-us/projects/chess/ you could start from there and traverse thru related work