4 ms·

This is the truth. Everyone wants AI to write their tests, but that’s backwards. AI will write the code and programmers will write the tests. The tests are the

This is the truth. Everyone wants AI to write their tests, but that’s backwards. AI will write the code and programmers will write the tests. The tests are the real business logic - binary assertions of how a system should or should not behave. The internals of how that happens are inconsequential.

zeroonetwothree 3y ago
Inconsequential until you start caring performance, scaling, reliability, interdependence with other systems, maintainability, or any other things that tests don’t assert.
- xboxnolifes 3y ago
  You can test performance, and you can monitor reliability. Scale and interdependence are spec requirements that are this theoretical AI's job. Maintainability is irrelevant to everyone here except the theoretical AI, as the AI does the maintaining.
  - ptx 3y ago
    This sounds a lot like some kind of government procurement process, which don't have a reputation for producing particularly good IT systems.
dontupvoteme 3y ago
Why not simply use AI to implement monitoring and evaluation so incredibly fine grained any human would go insane if they had to do it?
staunton 3y ago
I look forward to writing specifications ("property tests without RNG") in a theorem prover language and some AI writing the program against my specification.
nivenhuh 3y ago
I've used ChatGPT to write my tests. It's pretty effective at getting the first 5-6 use cases written out, but in a style that isn't correct. Copy/paste a code module, ask it to write the tests. Then, in the same context, you correct it by providing it with more information about how you like to structure the tests. It starts to understand the "style". Then, you start to provide the context with testing concerns. By the end of it, the AI context is pretty good (80-90'ish%). It's nice, as a developer, to be able to say: "test that asset is deleted when DELETE /asset/:id", and the AI converts that into a proper test case, using familiar variables / test structure / corrects the abbreviated text into a proper test title, etc..

zeroonetwothree 3y ago

Inconsequential until you start caring performance, scaling, reliability, interdependence with other systems, maintainability, or any other things that tests don’t assert.

xboxnolifes 3y ago
You can test performance, and you can monitor reliability. Scale and interdependence are spec requirements that are this theoretical AI's job. Maintainability is irrelevant to everyone here except the theoretical AI, as the AI does the maintaining.
- ptx 3y ago
  This sounds a lot like some kind of government procurement process, which don't have a reputation for producing particularly good IT systems.

dontupvoteme 3y ago

Why not simply use AI to implement monitoring and evaluation so incredibly fine grained any human would go insane if they had to do it?

staunton 3y ago

I look forward to writing specifications ("property tests without RNG") in a theorem prover language and some AI writing the program against my specification.

nivenhuh 3y ago

I've used ChatGPT to write my tests. It's pretty effective at getting the first 5-6 use cases written out, but in a style that isn't correct. Copy/paste a code module, ask it to write the tests. Then, in the same context, you correct it by providing it with more information about how you like to structure the tests. It starts to understand the "style". Then, you start to provide the context with testing concerns. By the end of it, the AI context is pretty good (80-90'ish%). It's nice, as a developer, to be able to say: "test that asset is deleted when DELETE /asset/:id", and the AI converts that into a proper test case, using familiar variables / test structure / corrects the abbreviated text into a proper test title, etc..