> March 2022. Cognitive scientist Gary Marcus publishes a scathing article: “Deep Learning Is Hitting a Wall.” [...] In March 2022, no less! Was he out of his m

> March 2022. Cognitive scientist Gary Marcus publishes a scathing article: “Deep Learning Is Hitting a Wall.” [...] In March 2022, no less! Was he out of his mind? That’s what the experts said back then—amid laughter and barely concealed scorn. Yet, with the evidence The Information has now presented, Marcus seems almost prophetic. Gary Marcus has been saying this since at least 2012: > Yet deep learning may well be approaching a wall, much as I anticipated earlier, at beginning of the resurgence (Marcus, 2012) To me it feels like having continually predicted rain throughout the longest dry-spell in history. Eventually you'll be right (assuming that we'll move to some other paradigm in due course), but it's hardly prophetic. I think that the challenges of intelligence and non-shallow reasoning will inherently involve fighting against diminishing returns (which is what the supposedly broken scaling laws predict) regardless of technique. Same as for computer graphics - doubling the compute doesn't double the perceptual quality, but that doesn't necessarily mean it's hitting a wall.