The way I see it is as expanding the high-level-low-level spectrum of programing tools towards the high end. In very large-scale projects that have many levels

The way I see it is as expanding the high-level-low-level spectrum of programing tools towards the high end. In very large-scale projects that have many levels of complexity, I would argue "probabilistic software" is already a reality. You don't even try to fix all the bugs or edge cases. You try to minimize the impact of failures while accepting that there will always be failures. Building in logic to fail more gracefully is a big part of that and cost/benefit of such efforts is a probabilistic question, even if it might rarely be framed as such. It's usually much easier (and never harder) to specify what needs to be done than how it should be done. Whether you can trust the result (enough) depends on the application. AI driven development will be applied first wheree errors and failures are least harmful and advance from there. It might take a long time until the degree of correctness improves enough and trust is build around it. After all, some countries' railroad networks still don't use computers but instead have a human map out new routes and schedules on paper to make sure trains don't crash. Nevertheless, the speed of AI development has exceeded (at least my) expectations time and time again in recent years. What might happen is that we eventually get a lot of software that fails more often than now but is a lot cheaper. That would still mean that the new methods are widely adopted. People accept software errors as a part of life already, so even if they get more frequent in less critical applications, we will adapt. What is really dangerous is when the various software components get too fast and complex to understand or control and develop pathological feedback loops in situations that cause real trouble. This kind of thing is a continuum of badness which tops out at "AI taking over and wiping out humanity". Given market incentives driving adoption, which I anticipate to be strong, it's hard to imagine how such risks might be mitigated.