Even if you assume the benefits here are all true and not overly optimistic, there is a significant downside. Software that is "better" by means of more complex

Even if you assume the benefits here are all true and not overly optimistic, there is a significant downside. Software that is "better" by means of more complexity via AI will be significantly harder for a human to understand. Even if it's generally more accurate, software that is a black box is inherently frustrating to at least some of it's users. If the advantage of AI generated software would be in solving problems through greater complexity than a human would think of, then it won't be able to achieve the kind of simplicity that makes it much easier to understand what software is actually doing.