I fully agree with the article that discussing sentience - at this point in time, at least - is not very useful. Perhaps it never will be when it comes to stati

I fully agree with the article that discussing sentience - at this point in time, at least - is not very useful. Perhaps it never will be when it comes to statistical models. That being said, there are at more than two academic factions discussing what deep neural networks do and don't do. And I can't help but feel like the symbolic crowd - Gary Marcus chief among them - have been sure about what these models do and don't do, only to be proven wrong repeatedly. At first these models were only fitting functions. Then language could never be learned due to its dimensionality. Then they were all only interpolating. Then they could never do math. etc. It just goes on and on. Sure, it's correct academic procedure that someone then goes and writes a paper showing that these models actually do certain things, or that it ain't so simple. And don't get me wrong, I don't even disagree that we will need more that just plain old' DNN's to throw data at to advance beyond certain points. In my view, the inductive bias of using Graph NN (Transformers) was simply one such step - and causal modeling may be another one. What irks me, however, is how sure the symbolists always are of every absolute truth they seem to know. No - in truth, we don't really know in what way these models interpolate or extrapolate, memorise data representations, fit functions or bases or polyhedra boundaries, or fail due to lack of inductive biases in task Y. And what's true changes each quarter year. At this point, symbolists keep making statements about what these models can't do, as they have for years. For me to take this seriously either way, I wanna see some mathematical proof. Because otherwise we are just going around in circles.