Just this morning I asked GPT-4o to write some code for me. And the code was correct, except for one stupid mistake GPT-4o made which caused a compilation error

Just this morning I asked GPT-4o to write some code for me. And the code was correct, except for one stupid mistake GPT-4o made which caused a compilation error. I just fixed that myself, but I think it is likely if I gave GPT-4o the error message it could have fixed it too. And I was thinking if I set up a chain-of-thought agent with function-calling, GPT-4o probably could have discovered and fixed the compilation error itself without my involvement. Provide it with unit tests it may even get the code to pass the tests (even if it takes a few iterations)-which could address issues like the braces in strings and comments issue you mention, assuming the unit tests cover them. And if they don’t-if you notice an issue via code inspection or exploratory testing, GPT-4o (in my experience) often does a decent job of “here is the code and here is a description of the bug, modify the code to fix it”. Of course, sometimes chain-of-thought agents get stuck and fail to progress, but something that quickly gives you the right answer 80% of the time can be a big productivity boost. In my case earlier today, it helped that it was a relatively simple function and I gave it a rather detailed natural language spec of what I wanted it to do. I totally could have written it all myself, but writing a natural language spec and getting GPT-4o to translate it to code is (depending on my mood) less mental effort than just writing the code directly.