It seems like observing usage and setting up a feedback loop, then incrementally a/b testing, is certainly not something that can be done today, but doesn't see

It seems like observing usage and setting up a feedback loop, then incrementally a/b testing, is certainly not something that can be done today, but doesn't seem beyond the realm of possibility a couple decades from now. Edit: In fact, now that I think about it, I'd probably be more ok dealing with a requirements document and testing plan generated by chatgpt today from a summary than I would be dealing with code -- hallucinations tend to have less impact at that stage. Requirements tend to be wrong anyways until they're implemented; code is where the rubber hits the road. The missing piece is having LLMs get better at integrating and reasoning about real world data, so that it can iterate on user testing. Heck, at that point, models can decide what to build, and not just how to build it; there's a treasure trove of data that they can access much faster than humans to find gaps in the market and generate solutions to fill those gaps. Assuming we don't hit a wall with model capabilities, we can focus purely on consuming the outputs of AI, as it optimizes itself to best keep us happy.