4 ms·

Excellent reply and I apologize for not having time to give you an equally thorough response at this time as I’m slammed. It’s such a worthy reply I apologize.

Excellent reply and I apologize for not having time to give you an equally thorough response at this time as I’m slammed. It’s such a worthy reply I apologize. But the question I would ask you is whether you are any barrier to simply scaling up the tokens on the existing technology? I don’t. I see us at the beginning of a ladder where the only input is capacity just like when Intel was young. While breakthroughs can change the path, the truth is we have a fairly predictable step-by-step to much more capable systems without one just by scaling the size of the hardware. Consider this argument: 1) I mean at core the crux of our learning is that predicting the next word may be what human thinking is generally about, and how our brain works, because that’s largely the innovation here. 2) Becoming better at doing that is entirely predictable and we can scale profoundly from current levels with hardware that we are putting into production right now and that we have already invented. 3) Therefore the path to next generation capabilities is relatively (and that’s important I admit) linear. So prior to breakthrough, we have a simple path forward to what would be at peast fairly advanced capabilities of language and media prediction and manipulation. Now your argument about progress is right. Predicting material progress of technology breakthroughs tends to be unpredictable and inherently dangerous, but we are in an accelerating trend and most of the time (big statement here, right?) the appearance of an acceleration trend tends to extend to continuance of the trend in a certain timeframe. At least that’s been the case with the “waves” of technology breakthrough since the Industrial Revolution according. I mean to invoke Smihula's theory of waves in this argument, since I know you’ll understand that. Those last two arguments are statistically supported and quite logical. How smart does it need to be and how statistically probable are excellent points. As an aside… For me, one of the areas I am focused on and thinking about a lot is self-organizing AI agents. Having worked a lot with large scale networks, agents and task specialized networked AI systems get me excited. My brain considers it a blue ocean opportunity. The parallels between human civilization density and current learning about density of population in demographics driving essential human progress makes me believe there will be parallels in AI. The more, the more they will self organize into network effects, and the outcome of this, like human civilization, will be high quality and rapid progress. I am not a believer in one gigantic AI, but networks of networks self organized in a way where they self-optimize around goals and outcomes, and we are really just at the beginning of exploring this direction of the technology. The biggest limiting factor to AI technology at this point is human input and the need for human oversight. While that oversight is definitely necessary, once AI becomes self organizing and self-creating, progress should be profound. Anyone who doesn’t think that’s going to happen needs to understand the nature of intelligence and realize it’s just a matter of time. You can’t go down this path in a meaningful way and repress only certain aspects of digital intelligence in the long term.

ben_w 3y ago
Thanks :) > But the question I would ask you is whether you are any barrier to simply scaling up the tokens on the existing technology? I don’t. I see us at the beginning of a ladder where the only input is capacity just like when Intel was young. My expectation is that we need algorithmic improvements rather than scaling; AI can read approximately all of the internet, but current models need to actually do so just to reach the level of intern or fresh graduate. While this makes them superhuman in the breadth of skills they can perform, they need something else to improve the maximum quality in any given skill — in some cases, we can already train them on synthetic data or self-play, e.g. chess, though I don't know how broad an impact that would have. But I do expect such algorithmic improvements, so in effect we are in agreement, if not in the details of how. When it comes to hardware improvements, I'm not sure how that particular landscape will change over the next decade. Transistors are close enough to atomic scale they can't go on much longer, and Dennard scaling has long since stopped, but that doesn't mean nobody cares or that nobody is working on the energy efficiency. And if — just if, it isn't necessarily true — if human level intelligence needs a network with as many free parameters as there are synapses in a human brain, we're around 3-4 orders of magnitude away from that at present. > I mean to invoke Smihula's theory of waves in this argument, since I know you’ll understand that. Thanks, I was unfamiliar with it: https://en.wikipedia.org/wiki/Smihula_waves https://en.wikipedia.org/wiki/Smihula_waves > Having worked a lot with large scale networks, agents and task specialized networked AI systems get me excited. My brain considers it a blue ocean opportunity. I think you're correct. The current zeitgeist is do-everything models, and the only blue ocean opportunities are found when you zig when everyone else is zagging, and vice-versa. Although, be quick; if my cursory reading of Smihula's theory of waves was correct, you don't have much time before the current market reaches saturation, and moves on to the next thing.
- ukuina 3y ago
  Smihula's waves, when extrapolated, would indicate a 15-year cycle for the smartphone era (assumed to have "innovation saturated" 2007-2022), and an 8-year cycle for AI (starting 2023). That's a staggering amount of change in too short a period to adjust to.

ben_w 3y ago

Thanks :) > But the question I would ask you is whether you are any barrier to simply scaling up the tokens on the existing technology? I don’t. I see us at the beginning of a ladder where the only input is capacity just like when Intel was young. My expectation is that we need algorithmic improvements rather than scaling; AI can read approximately all of the internet, but current models need to actually do so just to reach the level of intern or fresh graduate. While this makes them superhuman in the breadth of skills they can perform, they need something else to improve the maximum quality in any given skill — in some cases, we can already train them on synthetic data or self-play, e.g. chess, though I don't know how broad an impact that would have. But I do expect such algorithmic improvements, so in effect we are in agreement, if not in the details of how. When it comes to hardware improvements, I'm not sure how that particular landscape will change over the next decade. Transistors are close enough to atomic scale they can't go on much longer, and Dennard scaling has long since stopped, but that doesn't mean nobody cares or that nobody is working on the energy efficiency. And if — just if, it isn't necessarily true — if human level intelligence needs a network with as many free parameters as there are synapses in a human brain, we're around 3-4 orders of magnitude away from that at present. > I mean to invoke Smihula's theory of waves in this argument, since I know you’ll understand that. Thanks, I was unfamiliar with it: https://en.wikipedia.org/wiki/Smihula_waves https://en.wikipedia.org/wiki/Smihula_waves > Having worked a lot with large scale networks, agents and task specialized networked AI systems get me excited. My brain considers it a blue ocean opportunity. I think you're correct. The current zeitgeist is do-everything models, and the only blue ocean opportunities are found when you zig when everyone else is zagging, and vice-versa. Although, be quick; if my cursory reading of Smihula's theory of waves was correct, you don't have much time before the current market reaches saturation, and moves on to the next thing.

ukuina 3y ago
Smihula's waves, when extrapolated, would indicate a 15-year cycle for the smartphone era (assumed to have "innovation saturated" 2007-2022), and an 8-year cycle for AI (starting 2023). That's a staggering amount of change in too short a period to adjust to.