I did spend a little while evaluating what GPT could output for a fairly intensive gaming task - implement a particular novel physical dynamics model. (Yes, the

I did spend a little while evaluating what GPT could output for a fairly intensive gaming task - implement a particular novel physical dynamics model. (Yes, there are newer tools now: I haven't used them and don't want to pay to use them.) And what it did was impressive in a certain sense in that it wrote up a large body of code with believable elements, but closer examination demonstrated that, of course, it wasn't actually going to function as-is, it made up some functions, it left others stubbed. I would never trust it to manage memory or other system resources correctly, because AFAIK nobody has implemented that kind of "simulate the program lifecycle" bookkeeping into the generated output. There's a domain that I do find some application for it in, which is "configure ffmpeg". That and tasks within game engines that are similar in nature, where the problem to be solved involves API boilerplate, do benefit. What also works, to some degree, is to ask it to generate code that follows a certain routine spec, like a state machine. Again, very boilerplate-driven stuff, and I expect an 80% solution and am pleasantly surprised if it gets everything. And it is very willing to let me "define a DSL" on the fly and then act as a stochastic compiler, generating output patterns that are largely in expectation with the input. That is something I know I could explore further. And I'm OK with this degree of utility. I think I can refine my use of it, and the tools themselves can also be refined. But there's a ceiling on it as a directly impactful thing, where the full story will play out over time as a communication tech, not a "generate code" tech, with the potential to better address Conway's law development issues.