2 ms·

> If you expand the time frame to 10-20 years, then AI is poised to become something that truly upsets most of the economy. Obviously, nothing can be said for c

> If you expand the time frame to 10-20 years, then AI is poised to become something that truly upsets most of the economy. Obviously, nothing can be said for certain, there could be black swan events, etc. But on the current trajectory [...] This is an interesting framing that I think exposes a key difference between people who view AI dominance as inevitable and those who don't. You view the progress of AI slowing down as a black swan event, something that's extremely unlikely. I, on the other hand, view it as inevitable. Technology has never progressed along a single axis linearly or exponentially for very long. Technology progresses in bursts—there's a major breakthrough that leads to a huge amount of interest and progress which leads to rapid growth in a particular area until it saturates. After a few years we start to see diminishing returns as the area opened by the breakthrough becomes fully explored, and eventually progress becomes very slow indeed until the next major breakthrough. Given this general pattern of progress in technology, it would be very foolish indeed to predict the next 20 years' trajectory of AI less than 2 years after ChatGPT was released.

therobots927 2y ago
You worded your comment much better than I did, and you touch on the key point in all of this which is the assumption that chatGPT etc will continue to get noticeably better at regular intervals. It’s entirely possible that openAI has reached a local maximum with the data, compute, and model structure they have available. The data is tapped out for the most part. They already probably acquired 80% of the available text training data. Compute is now heavily cost constrained. This is obvious because even GPTs responses have gotten worse over the last 6 months, especially with image generation tasks. Likely because they’re throttling compute. Model structure: OpenAI has admitted that they cannot understand why chatGPT has generated a given output. If you can’t explain your model you can’t improve it either. And more compute won’t help you that much here if I had to guess.
HarHarVeryFunny 2y ago
I also think human level (and beyond) AGI is inevitable, but that in of itself says nothing about the time frame in which it will be achieved. People have been predicting AI/AGI ever since the advent of computers over 70 years ago, and estimates have always been way over-optimistic. It's perhaps relevant to note that the reason estimates have always been off seems to be that people have always thought that the technology of the day was sufficient, and that it was just a matter of applying it. So far we've had procedural programming (SHRLDU), expert systems (CYC), general-purpose problem solvers (SOAR), PDP/connectionist approaches, modern neural nets, LLMs... I do think we're getting closer, and that neural nets will get us there (maybe not with gradient descent as the learning method though). There have been lots of valuable lessons and insights learned from various neural net architecture, not least of which is the confirmation by LLMs that prediction seems to be the basis (or at least one of major pillars) of intelligence. And yet, I can't help but feel that there is still a long way to go ...

therobots927 2y ago

You worded your comment much better than I did, and you touch on the key point in all of this which is the assumption that chatGPT etc will continue to get noticeably better at regular intervals. It’s entirely possible that openAI has reached a local maximum with the data, compute, and model structure they have available. The data is tapped out for the most part. They already probably acquired 80% of the available text training data. Compute is now heavily cost constrained. This is obvious because even GPTs responses have gotten worse over the last 6 months, especially with image generation tasks. Likely because they’re throttling compute. Model structure: OpenAI has admitted that they cannot understand why chatGPT has generated a given output. If you can’t explain your model you can’t improve it either. And more compute won’t help you that much here if I had to guess.

HarHarVeryFunny 2y ago

I also think human level (and beyond) AGI is inevitable, but that in of itself says nothing about the time frame in which it will be achieved. People have been predicting AI/AGI ever since the advent of computers over 70 years ago, and estimates have always been way over-optimistic. It's perhaps relevant to note that the reason estimates have always been off seems to be that people have always thought that the technology of the day was sufficient, and that it was just a matter of applying it. So far we've had procedural programming (SHRLDU), expert systems (CYC), general-purpose problem solvers (SOAR), PDP/connectionist approaches, modern neural nets, LLMs... I do think we're getting closer, and that neural nets will get us there (maybe not with gradient descent as the learning method though). There have been lots of valuable lessons and insights learned from various neural net architecture, not least of which is the confirmation by LLMs that prediction seems to be the basis (or at least one of major pillars) of intelligence. And yet, I can't help but feel that there is still a long way to go ...