4 ms·

Less thinking. More testing.

10 points by chrismealy 16y ago

delano 16y ago
"Talk doesn't cook rice."
10ren 16y ago
Because this article covers a lot of ground, it really struck me that the term testing is ambiguous: it can mean unit-testing or user-testing. In prototyping sprints, TDD seems to acknowledge that unit-testing slows you down and creates rigidity, and therefore encourages you to just get the basic thing working without unit tests. You can throw it away, or fundamentally change the outcome, or the architecture, or the structure of the code. User-testing is how you test a prototype (including just playing with it yourself). But in building a codebase that you will live with for sometime, that will be built-on and modified and bug-fixed and tweaked, unit-testing becomes really helpful.
xenophanes 16y ago
> I no longer trust long chains of reasoning from seemingly self-evident low-level principles. The correct principle is not to "trust" long chains of reasoning which one doesn't understand in their entirety, especially ones with strong conclusions. The problem here isn't long chains of reasoning (we could hardly do without any of those), but rather failure to hold them to high standards in how understandable they are, and failure to put in the effort to understand the whole thing before accepting it.
- randallsquared 16y ago
  Quite often a person believes they understand something when they don't. In this case, no amount of exhortation to understand the chain of reasoning will help. Unfortunately, I think there's a false dichotomy here; you have to understand the problem well enough to write useful tests, and if you don't understand a failure mode, you're probably not going to write anything to test it.
joubert 16y ago
I had a thought along the following lines: if I have test cases for a feature, it would be really cool if I have an engine which can figure out code to satisfy thoses tests. I.e. I define the contract; the computer figures out how to solve. Kinda like Simonyi's intentional programming. Is anybody working on something in this area?
- GeoffWozniak 16y ago
  I did a thesis that explored aspects of generating code that passed tests, but it was limited to deriving adequate structural representations based on behavioural specifications. Two approaches were shown: changing data structure representations to meet requirements and generating class structure/hierarchy. The thesis showed that it is possible to do this and how to do it, but it is a huge topic. If you're interested, I'll put more effort into making my thesis available online in HTML format. In the meantime, you can get it in a variety of places. http://duckduckgo.com/?q=Structuring%20Data%20via%20Behavioural%20Synthesis http://duckduckgo.com/?q=Structuring%20Data%20via%20Behaviou... Incidentally, generating behavioural elements (like entire functions) to pass tests would be very difficult although I have been toying with ways to try it out.
- gjm11 16y ago
  I don't know about that (it seems like doing it in general would be impossible, but some easy cases might be manageable), but there's a nice converse: QuickCheck (http://www.cs.chalmers.se/~rjmh/QuickCheck/ http://www.cs.chalmers.se/~rjmh/QuickCheck/) takes Haskell programs with contracts and effectively generates its own test suites.
stretchwithme 16y ago
creating a situation where users are separated into two different groups and then making a single variable different for each groups can indicate the effects of the variable given a large enough group size but taking that experience and applying it to history where many things are changing and where you cannot separate everyone into 2 different groups will not necessarily tell you anything about one arbitrarily selected variable.

10 points by chrismealy 16y ago

"Talk doesn't cook rice."

10ren 16y ago

Because this article covers a lot of ground, it really struck me that the term testing is ambiguous: it can mean unit-testing or user-testing. In prototyping sprints, TDD seems to acknowledge that unit-testing slows you down and creates rigidity, and therefore encourages you to just get the basic thing working without unit tests. You can throw it away, or fundamentally change the outcome, or the architecture, or the structure of the code. User-testing is how you test a prototype (including just playing with it yourself). But in building a codebase that you will live with for sometime, that will be built-on and modified and bug-fixed and tweaked, unit-testing becomes really helpful.

xenophanes 16y ago

> I no longer trust long chains of reasoning from seemingly self-evident low-level principles. The correct principle is not to "trust" long chains of reasoning which one doesn't understand in their entirety, especially ones with strong conclusions. The problem here isn't long chains of reasoning (we could hardly do without any of those), but rather failure to hold them to high standards in how understandable they are, and failure to put in the effort to understand the whole thing before accepting it.

randallsquared 16y ago
Quite often a person believes they understand something when they don't. In this case, no amount of exhortation to understand the chain of reasoning will help. Unfortunately, I think there's a false dichotomy here; you have to understand the problem well enough to write useful tests, and if you don't understand a failure mode, you're probably not going to write anything to test it.

joubert 16y ago

I had a thought along the following lines: if I have test cases for a feature, it would be really cool if I have an engine which can figure out code to satisfy thoses tests. I.e. I define the contract; the computer figures out how to solve. Kinda like Simonyi's intentional programming. Is anybody working on something in this area?

GeoffWozniak 16y ago
I did a thesis that explored aspects of generating code that passed tests, but it was limited to deriving adequate structural representations based on behavioural specifications. Two approaches were shown: changing data structure representations to meet requirements and generating class structure/hierarchy. The thesis showed that it is possible to do this and how to do it, but it is a huge topic. If you're interested, I'll put more effort into making my thesis available online in HTML format. In the meantime, you can get it in a variety of places. http://duckduckgo.com/?q=Structuring%20Data%20via%20Behavioural%20Synthesis http://duckduckgo.com/?q=Structuring%20Data%20via%20Behaviou... Incidentally, generating behavioural elements (like entire functions) to pass tests would be very difficult although I have been toying with ways to try it out.
gjm11 16y ago
I don't know about that (it seems like doing it in general would be impossible, but some easy cases might be manageable), but there's a nice converse: QuickCheck (http://www.cs.chalmers.se/~rjmh/QuickCheck/ http://www.cs.chalmers.se/~rjmh/QuickCheck/) takes Haskell programs with contracts and effectively generates its own test suites.

stretchwithme 16y ago

creating a situation where users are separated into two different groups and then making a single variable different for each groups can indicate the effects of the variable given a large enough group size but taking that experience and applying it to history where many things are changing and where you cannot separate everyone into 2 different groups will not necessarily tell you anything about one arbitrarily selected variable.