What a bizarre and disappointing article. The problem with this whole field is lop-sided expertise. Either neuroscience and no theoretical compusci, or compusci

What a bizarre and disappointing article. The problem with this whole field is lop-sided expertise. Either neuroscience and no theoretical compusci, or compusci and no stats; or stats and compusci, but no empiricism. > Large language models really do approximate the way neurons work, at least at the interface level. This shows a profound misunderstanding of the mathematics underneath NNs. To be trained, eg., on character strings in books... sets the structure of the NN. This is the only relevant property of NNs to consider in any homology with any brain. Brain structure isnt set by character sequence positioning, even if "somehow, very abstractly" backprop with gradient descent were how "brains worked" (an insane claim, now wholey discredited). The only sense in which an NN could even possibly be structurally homologous to the brain were if it were trained on actual brain structure data. Since, at the outside, we know that this training doesnt produce homology.. all we're talking about is a confusion which arises when naming interpolation "learning". > so let's stop pretending that we do Again, quite disappointing. We do know far more than enough to say that compressing a million books doesnt produce sentience. We know what criteria an intelligent system has to have: it has to be responsive to its environment with reasons that the environment alone under-determines. It has to fail well. It has to cope. Intelligence is not the production of "the right answer". Intelligence is the strategy of animals given ignorance about the world and the lack of "the right answer" to hand. Intelligence is what writes all those books which these systems (almost) wrote learn. Intelligence is that capacity to be in the world, to have something to say about it, to fail to describe it and yet succeed in getting close. Intelligence is formulation of theories in the absense of relevant data. Intelligence is a survival strategy. We know vast amounts about we animals, and what we are doing. We know vast amounts about theoretical computer science. And likewise about stats, and NNs. And we know that training a regression model on sequences of characters does not produce a system with something to say about the world. It has no intentions; it has never been anywhere. It has never tried to say anything. When it says "I like new york" it does so only because those characters have an order we wrote. It doesnt do so because it has been to new york, has a taste for pizza, and has enjoyed its time there. To impart text generation systems with sentience, systems which generate text because it has been written before, not because, the system intends to mean what it say -- is psychosis. And what we're seeing here is techno-utopian charlatans conscripting a willing cohort of people on the edges of sanity into a new astrology. We need to be extremely careful here: most people arent sceptics; esp. most "sceptics".