3 ms·
The current generation's interest is indeed quite understandably focused on, pardon the simplification, data-driven correlation engines and the results have bee
by whyowhy3484939 2y ago
The current generation's interest is indeed quite understandably focused on, pardon the simplification, data-driven correlation engines and the results have been nothing but spectacular. Let it be clear that I do not reject the idea(s) behind the bitter lesson. On the contrary, I think it is spot on, yet I feel there is in a great many instances a false, mostly unspoken, dichotomy being imagined behind those words: there is either A) raw probabilistic association lacking any and all structure or B) full scale good old fashioned symbolic AI containing nothing but highly specific, human, knowledge.
The text of the bitter lesson itself provides an interesting case, on the topic of computer vision: "Early methods conceived of vision as searching for edges, or generalized cylinders, or in terms of SIFT features. But today all this is discarded. Modern deep-learning neural networks use only the notions of convolution and certain kinds of invariances, and perform much better."
To me this proves the necessity of proper measurements devices - in this case "convolution" and "certain kinds of invariances" - to render data as intelligible as possible utilizing as little moving parts as required. What I don't see in this is: "we need to drop all preexisting notions of structure and just focus on finding pure correlation only". The fact that we have moved to more fundamental search operations - from "cylinder" to "convolution" - does not IMO fundamentally alter the general strategy. In short, I think the lesson is being stretched beyond its limits.
The lesson shows us: "We want AI agents that can discover like we can, not which contain what we have discovered" and I couldn't agree more, yet we seems to disagree on what it means to "discover like we can". I happen to believe that raw correlation is just one of the key ingredients. In fact Kant has the notion of "community" that, and I might be sorely mistaken here, appears to capture the associative power of cognition in much the same way.
I, of course, lack any sort of credentials to further my point. I only imagine it might possibly be of slight interest to someone to squint upon this topic from a skewed angle.
PS: Your point on pragmatism is well taken. I'm sure we are all quite excited to see what happens over the next couple of years, yet I cannot help but hear an ominous violin crescendo beneath your "if the predictions are consistent and reliable".
- QuesnayJr 2y agoThere has been considerable progress since Sutton wrote. The latest models don't even need convolutions and certain kinds of invariances. Visual Transformer models don't even necessarily know which pixels are near each other. They figure it out through correlations. I was skeptical of the statistical approach myself, but I think it's time for some humility in the face of its wild success.