4 ms·

What would it look like if a tool like this was taken to its logical conclusion: an agent that could write unit tests for any existing code and get it to 100% c

by uludag 2y ago

What would it look like if a tool like this was taken to its logical conclusion: an agent that could write unit tests for any existing code and get it to 100% code coverage. Not only that, it would have tests for every combinatorial code path reachable. Test files would outnumber human code by orders of magnitude. The ultimate pipe dream of some software developers would be fulfilled. Would such a tool be helpful? probably in some circumstances (e.g. spacecraft software perhaps), but I sure wouldn't want such a tool. If this is less than ideal, then how do we reach a compromise? What code branches and functions should be left untested? Is this question even answerable from the textual representation of code and documentation?

malfist 2y ago
In my opinion, no. AI writing unit tests can't know the original intent is the code, only the written intent. So an AI written test case gives you a false sense of security. Additionally, having such high coverage amounts often adds friction to writing new code, for good or bad
keybored 2y ago
Eventually you have so many tests that any change at all will break some of them. So you ask someone if the test makes sense or should be removed. But there is no one to ask.
tivert 2y ago
> What would it look like if a tool like this was taken to its logical conclusion: an agent that could write unit tests for any existing code and get it to 100% code coverage. Not only that, it would have tests for every combinatorial code path reachable. Test files would outnumber human code by orders of magnitude. The ultimate pipe dream of some software developers would be fulfilled. That dream is a nightmare. All code has bugs and that test suite would remove any "not bug" signals provided by the test cases. Most if not all utopias turn out to by dystopias if you look hard enough.

by uludag 2y ago

malfist 2y ago

In my opinion, no. AI writing unit tests can't know the original intent is the code, only the written intent. So an AI written test case gives you a false sense of security. Additionally, having such high coverage amounts often adds friction to writing new code, for good or bad

keybored 2y ago

Eventually you have so many tests that any change at all will break some of them. So you ask someone if the test makes sense or should be removed. But there is no one to ask.

tivert 2y ago

> What would it look like if a tool like this was taken to its logical conclusion: an agent that could write unit tests for any existing code and get it to 100% code coverage. Not only that, it would have tests for every combinatorial code path reachable. Test files would outnumber human code by orders of magnitude. The ultimate pipe dream of some software developers would be fulfilled. That dream is a nightmare. All code has bugs and that test suite would remove any "not bug" signals provided by the test cases. Most if not all utopias turn out to by dystopias if you look hard enough.