[NOT THE OP] >> All you see is numbers and functions, and all I see is problems being solved by these AI/ML methods and often in quantifiably better ways than

[NOT THE OP] >> All you see is numbers and functions, and all I see is problems being solved by these AI/ML methods and often in quantifiably better ways than intelligent humans can. Maybe in playing chess and go, but in any other task that neural networks have been compared to humans directly, the ability to outperform humans is more informative of the limitations of the measures of performance, than of the capabilities of the tested systems (typically, deep neural networks). Take large language models (LLMs) for example. Last time I looked -it was a while ago but- LLMs like BERT, ERNIE and their famous friends scored very highly and even above humans in natural language understanding tasks, such as those in the GLUE and SUPERGLUE benchmarks. You'd think that a system that scores 85% in a language understanding task has developed at least _some_ ability to understand language. And yet this is not the case. LLMs, despite their ability to score highly in formal benchmarks, remain dumb as bricks and as incapable of understanding human language as a rock. We know this because every once in a while someone in the field of NLP looks away from their favourite leaderboard for long enough to ask serious questions about the performance of systems, as for example [1] who started by asking why a language model like BERT would be any good at argument comprehension (BERT scored only three points below humans in a relevant benchmark task). Everytime anyone undertakes this kind of analysis of the abiltiy of LLMs to understand language, the conclusion is that they don't, and instead they learn "cheap tricks" and overfit to statistical regularities of the benchmark datasets. Similar analyses have found similar results in machine vision (e.g. [2], [3]) etc. The question then is, in what sense, exactly, is an automated system solving problems "in quantifiably better ways than intelligent humans can", when that system is not even solving the problem it's supposed to be solving, like the problem of understanding language by LLMs? The idea that those systems "solve problems", when they 're really only overfitting to their training sets, is an artifact of the shoddy scholarship that is typical in deep learning research, and the OP is right to be concerned that there is little understanding between machine learning researchers of what, exactly, it is that their systems are doing- or even trying to do. And, as the OP points out, I'm afraid that really is the result of a poor background, not only in sciences and mathematics, but in AI and even machine learning itself, among the current generation of machine learning researchers. Contrary to what the OP says, though, this paucity of relevant background is the result of an influx to the field of people from the sciences, who have a background in continuous mathematics, but not in AI or computer science. It is, indeed, a background in computer science that can best guard against misconceptions about computers being wonder machines with magickal intelligence abilities. At the very least, you don't have to explain to a computer scientist why solving one instance of a class of problems does not mean a general solution for the entire class. That comes built-in to most who have graduated in the discipline. But is it built-in to phycisists, or biologists, who study very different systems? ________________ [1] Probing Neural Network Comprehension of Natural Language Arguments https://aclanthology.org/P19-1459/ https://aclanthology.org/P19-1459/ [2] Intriguing properties of neural networks https://research.google/pubs/pub42503/ https://research.google/pubs/pub42503/ [3] The Elephant in the Room https://arxiv.org/abs/1808.03305 https://arxiv.org/abs/1808.03305