3 ms·

I think that automated testing is superior to any other tool/technique that we can use to avoid mistakes in programming. Can anyone argue with this? What is the

I think that automated testing is superior to any other tool/technique that we can use to avoid mistakes in programming. Can anyone argue with this? What is the alternative?

bad_user 9y ago
Static typing combined with functional programming is superior to automated testing, although the three are actually complementary, because automatic, property-based testing works better if you have type info (such that the library knows what values to generate) and referential transparency, because side effects can be awkward to specify as laws. So it is not a coincidence that Haskell developers have innovated testing tools and are now sold on QuickCheck, those two working well together ;-) Being the Scala developer that I am, I love ScalaCheck, but I’ve also used jsverify for JavaScript ... not as good as ScalaCheck or QuickCheck, due to not having static typing and type classes btw, but did the job.
- mishkovski 9y ago
  Static typing or functional programming can protect you only from some types of problems. With automated tests you have the flexibility to define the problem you are verifying your code against.
  - lkitching 9y ago
    By the same token, tests can only verify certain properties of a function. How could you test that a function is free from side-effects? How could you test it never returns null?
- hwayne 9y ago
  QuickCheck is actually older than the TDD movement!
Jtsummers 9y ago
There is no best tool or technique because there is no silver bullet. And what do you mean by automated testing? What kind of tests? Unit, integration, regression, black box, white box? Automated testing is only one tool, that every system should be using. Manual testing is obviously too tedious (though sometimes required) to do for all test sequences for most systems so of course we need to automate. But what do we test? How do we develop the tests? Do we test type invariance constraints? In a dynamically typed language you absolutely should! In a statically typed language, you might need to. In C I need to test that my int foo is within the expected range. In Ada I can define a new type Foo which will always be within the right range. Both are statically typed, one offers more than the other. How do we define a unit? Do I want tests of every class, of every function? Do I write unit tests for private portions of classes? What should be hidden from users as "implementation details", but actually matters to me as the implementer. How do we actually test out concurrency bugs? I might have a concurrency bug that shows up in 1 of 20 potential traces of the concurrent system. But practically only shows up 1 time in 1,000,000. That's impossible to show with tests with any reliability, but almost trivial to show with TLA+ and some other formal methods (if you can abstract the concurrent algorithm properly). Maybe the bug only occurs when I have at least 10 users on the system, but is still rare or hard to force into existence. The alternative: Learn static typing. Learn math. Learn formal methods. Learn multiple concurrency paradigms. Learn multiple languages. Internalize their concepts. When you write a program, design the program. Maybe not a full formal spec, but you need to know what it should do or you cannot verify it. You cannot have any confidence in it. You use a dynamic language like Ruby? Guess what, you can make it do something like Ada's type system! Create a new class instead of using a generic integer that redefines + and - and other ops so that you get the desired properties (like range constraints or modular arithmetic) but still otherwise acts like an integer. You use a language like C that lets you clobber all over the memory in other threads. Great, learn about channels, semaphores, and message queues. Pick the appropriate subset (or library if viable) and design your concurrent algorithms in a disciplined fashion.
- mishkovski 9y ago
  I meant automated testing(any kind) vs no automated testing. Most of us don't do any kind of automated testing.

I think that automated testing is superior to any other tool/technique that we can use to avoid mistakes in programming. Can anyone argue with this? What is the alternative?

bad_user 9y ago

Static typing combined with functional programming is superior to automated testing, although the three are actually complementary, because automatic, property-based testing works better if you have type info (such that the library knows what values to generate) and referential transparency, because side effects can be awkward to specify as laws. So it is not a coincidence that Haskell developers have innovated testing tools and are now sold on QuickCheck, those two working well together ;-) Being the Scala developer that I am, I love ScalaCheck, but I’ve also used jsverify for JavaScript ... not as good as ScalaCheck or QuickCheck, due to not having static typing and type classes btw, but did the job.

mishkovski 9y ago
Static typing or functional programming can protect you only from some types of problems. With automated tests you have the flexibility to define the problem you are verifying your code against.
- lkitching 9y ago
  By the same token, tests can only verify certain properties of a function. How could you test that a function is free from side-effects? How could you test it never returns null?
hwayne 9y ago
QuickCheck is actually older than the TDD movement!

Jtsummers 9y ago

There is no best tool or technique because there is no silver bullet. And what do you mean by automated testing? What kind of tests? Unit, integration, regression, black box, white box? Automated testing is only one tool, that every system should be using. Manual testing is obviously too tedious (though sometimes required) to do for all test sequences for most systems so of course we need to automate. But what do we test? How do we develop the tests? Do we test type invariance constraints? In a dynamically typed language you absolutely should! In a statically typed language, you might need to. In C I need to test that my int foo is within the expected range. In Ada I can define a new type Foo which will always be within the right range. Both are statically typed, one offers more than the other. How do we define a unit? Do I want tests of every class, of every function? Do I write unit tests for private portions of classes? What should be hidden from users as "implementation details", but actually matters to me as the implementer. How do we actually test out concurrency bugs? I might have a concurrency bug that shows up in 1 of 20 potential traces of the concurrent system. But practically only shows up 1 time in 1,000,000. That's impossible to show with tests with any reliability, but almost trivial to show with TLA+ and some other formal methods (if you can abstract the concurrent algorithm properly). Maybe the bug only occurs when I have at least 10 users on the system, but is still rare or hard to force into existence. The alternative: Learn static typing. Learn math. Learn formal methods. Learn multiple concurrency paradigms. Learn multiple languages. Internalize their concepts. When you write a program, design the program. Maybe not a full formal spec, but you need to know what it should do or you cannot verify it. You cannot have any confidence in it. You use a dynamic language like Ruby? Guess what, you can make it do something like Ada's type system! Create a new class instead of using a generic integer that redefines + and - and other ops so that you get the desired properties (like range constraints or modular arithmetic) but still otherwise acts like an integer. You use a language like C that lets you clobber all over the memory in other threads. Great, learn about channels, semaphores, and message queues. Pick the appropriate subset (or library if viable) and design your concurrent algorithms in a disciplined fashion.

mishkovski 9y ago
I meant automated testing(any kind) vs no automated testing. Most of us don't do any kind of automated testing.