> Eerke Boiten, Professor of Cyber Security at De Montfort University Leicester, explains his belief that current AI should not be used for serious applications

> Eerke Boiten, Professor of Cyber Security at De Montfort University Leicester, explains his belief that current AI should not be used for serious applications. > In my mind, all this puts even state-of-the-art current AI systems in a position where professional responsibility dictates the avoidance of them in any serious application. > Current AI systems also have a role to play as components of larger systems in limited scopes where their potentially erroneous outputs can be reliably detected and managed, or in contexts such as weather prediction where we had always expected stochastic predictions rather than certainty. I think it's important to note that: - Boiten is a security expert, but doesn't have a background working in ML/AI - He never defines what "serious application" means, but apparently systems that are designed to be tolerant of missed predictions are not "serious". He seems to want to trust a system at the same level that he trusts a theorem proved with formal methods, etc. I think the frustrating part of this article is that from a security perspective, he's probably right about his recommendations, but he seems off-base in the analysis that gets him there. > Current AI systems have no internal structure that relates meaningfully to their functionality. They cannot be developed, or reused, as components. Obviously AI systems do have internal structure, and there are re-usable components both at the system level (e.g. we pick an embedding, we populate some vector DB with contents using that embedding, and create a retrieval system that can be used in multiple ways). The architecture of models themselves also has components which are reused, and we make choices about when to keep them frozen versus when to retrain them. Any look at architecture diagrams in ML papers shows one level of these components. > exponential increases of training data and effort will give us modest increases in impressive plausibility but no foundational increase in reliability. I think really the problem is that we're fixated on mostly-solving an ever broader set of problems rather than solving the existing problems more reliably. There's plenty of results about ensembling and learning theory that give us a direction to increase increase reliability (by paying for more models of the same size), but we seem far more interested in seeing if we can most of the time solve problems at a higher level of sophistication. That's a choice that we're making. Similarly Boiten mentions the possibility of models with explicit confidences -- and there's been plenty of work on that but b/c there's a tradeoff with model size (i.e. do you want to spend your resources on a bigger model, or on explicitly representing variance around a smaller set number of parameters?) but people seem mostly uninterested. I think there are real reasons to be concerned about the specific path we're on, but these aren't the good ones.