5 ms·

You are definitely on to something here, but the difference is that the fundamental process was proven. It "just" needed to scale. That's hard and complex, but

You are definitely on to something here, but the difference is that the fundamental process was proven. It "just" needed to scale. That's hard and complex, but on a different level. I don't think anyone doubted the nature of the technology. The bits were being sent. It's not like we were unsure of the fundamental possibility of transmitting information. The potential was shown very, very early on (Mother of all demos was in 1968). What we were and to some extent still are unsure of is the practical impact on society. AI and LLMs in particular are not even at the mother of all demos level yet notwithstanding the grandiose claims and demos. There is no consensus on what these models are even doing. There is (IMO) justified skepticism surrounding the claims of reasoning and ability to abstract. We are in my opinion not yet at the "bits are being sent" stage.

datahack 2y ago
I see this as entirely surmountable. We’re still making geometric progress in small model accuracy, and breakthroughs like test-time training and synthetic data are poised to deliver immediate gains in self-training performance. Your point about skepticism being warranted when viewing this linearly is well taken. But this isn’t a linear path. The Internet, at its core, was about connecting computers to unlock the value of those connections—a transformative but relatively straightforward concept. What we’re dealing with now is the training of cognitive digital intelligence. This is an inherently dynamic and breakthrough-oriented process, one that evolves in ways far less predictable or constrained than simple network effects. While the metaphor of connectivity is useful, it doesn’t fully capture the parallel, multi-dimensional approaches at play here. Pessimism, in my view, is deeply unwarranted, especially given the history of technological progress. Time and again, advancements have proven to be far more impactful and beneficial than even the most optimistic predictions. Consider the projections for AI in 2017—most futurists undershot its actual progress by an order of magnitude. This research clearly illuminates a path forward: https://ekinakyurek.github.io/papers/ttt.pdf https://ekinakyurek.github.io/papers/ttt.pdf Deeply appreciate your thoughtful comment.

datahack 2y ago

I see this as entirely surmountable. We’re still making geometric progress in small model accuracy, and breakthroughs like test-time training and synthetic data are poised to deliver immediate gains in self-training performance. Your point about skepticism being warranted when viewing this linearly is well taken. But this isn’t a linear path. The Internet, at its core, was about connecting computers to unlock the value of those connections—a transformative but relatively straightforward concept. What we’re dealing with now is the training of cognitive digital intelligence. This is an inherently dynamic and breakthrough-oriented process, one that evolves in ways far less predictable or constrained than simple network effects. While the metaphor of connectivity is useful, it doesn’t fully capture the parallel, multi-dimensional approaches at play here. Pessimism, in my view, is deeply unwarranted, especially given the history of technological progress. Time and again, advancements have proven to be far more impactful and beneficial than even the most optimistic predictions. Consider the projections for AI in 2017—most futurists undershot its actual progress by an order of magnitude. This research clearly illuminates a path forward: https://ekinakyurek.github.io/papers/ttt.pdf https://ekinakyurek.github.io/papers/ttt.pdf Deeply appreciate your thoughtful comment.