3 ms·
ChatGPT tends to over mock the module to the point it will always pass the test, so except for utilities module, this isn't helping much either. This is especia
by NhanH 3y ago
ChatGPT tends to over mock the module to the point it will always pass the test, so except for utilities module, this isn't helping much either. This is especially common in JS environment.
If you think about it, it does make sense why: the prompt tends to have no where near the context of how the module should be delineated. Normally the programmer knows how the abstraction within his program is separated, but we have no good way to tell it to chatGPT. That means unless your program structure is still following one of the common structure and hasn't morphed into your business domain structure, ChatGPT will mock the wrong layer/ abstraction in the test suite.
- jstarfish 3y agoI don't know JS, but it works pretty well for me in Python. Where it goes astray, I'll put comments indicating what it's supposed to do and GPT tends to pick up on it. It's like instruction-based inpainting/outpainting. The biggest hurdle is context limit, especially from retained comments. The real benefit to my system is that it weeds out the zero-effort clowns by forcing them to validate their own work before I waste my time trying to do anything with it.