I find this kind of argument comes up a lot and it seems fundamentally flawed to me. 1. You can set a bar wherever you want for a level of "seriousness" and hu

I find this kind of argument comes up a lot and it seems fundamentally flawed to me. 1. You can set a bar wherever you want for a level of "seriousness" and huge swathes of real world work will fall below it, and are therefore attractive to tackle with these systems. 2. We build critical large scale systems out of humans, which are fallible and unverifiable. That's not to say current LLMs are human or equivalent, but "we can't verify X works all the time" doesn't stop us doing exactly that a lot. We deal with this by learning how humans make mistakes, why, and build systems of checks around that. There is nothing in my ind that stops us doing the same with other AI systems. 3. Software is written by, checked by and verified by humans at least at some critical point - so even verified software still has this same problem. We've also been doing this kind of thing with ML models for ages, and we use buggy systems for an enormous amount of work worldwide. You can argue we shouldn't and should have fully formally verified systems for everything, but you can't deny that right now we have large serious systems without that. And if your goal is "replace a human" then I just don't think you can reasonably say that it requires verifiable software. > Systems are not explainable, as they have no model of knowledge and no representation of any ‘reasoning’. Neither of those statements are true are they? There are internal models, and recent models are designed around having a representation of reasoning before replying. > current generative AI systems represent a dead end, where exponential increases of training data and effort will give us modest increases in impressive plausibility but no foundational increase in reliability And yet reliability is something we see improve as LLMs get better and we get better at training them.