The first fallacy is that you can't "write" a test, because a test is an event, not an artifact. What coders mean when they say they "write a test" is that they

The first fallacy is that you can't "write" a test, because a test is an event, not an artifact. What coders mean when they say they "write a test" is that they have written an output check. You can write a test procedure, but the test procedure is not the test itself. If you sit down and watch a tester in the act of testing (try videoing it and analyzing 2 sec. slices, while labeling everything that is done, as I have) you will see that testing is a very rich process. It has a heuristic, rather than an algorithmic nature. It involves substantial social competence and judgment, which is a tacit skill set. It is not encodeable. The next problem is that ChatGPT is not thinking critically about risk. It thinks (well, no it doesn't think... let's say it "treats")... it treats testing as a process of demonstration. Demonstration is not testing. Demonstration proves that functionality is possible, but not that it is reliable. Thus, it produces checking code that is shallow. The next problem is how are you going to explain to ChatGPT what your product does? Do you give it all your source code? Do you give it all your Jira tickets? The demos I have seen are toy examples. Nothing on a realistic scale. Let's say you feed it a whole spec and source code, somehow. The next problem is omissions. ChatGPT arbitrarily stops producing output. You then have to carefully check its work to see what it has left out. What you are asking ChatGPT to do, really, is to write code that will maximize the probability of spotting a real bug, while minimizing the probability of a false positive. But it will arbitrarily focus on only those kinds of bugs that it can easily discover. This is the oracle problem. The next problem is that ChatGPT halucinates or misunderstands, so you have to correct its mistakes. Sometimes going through frustrating iterations of prompts, like a man trying to pull a mule through town. You also have trouble with test data and data setup. Only in toy examples is this not a significant problem. ChatGPT produces conventional results, and that does have some value. But it's not enough for professional work. I just think the people who casually say that ChatGPT is going to easily handle these things are pretty stupid, under the definition of stupidity as "the refusal to think."