4 ms·

What kind of time frame is "any time soon"? If you're thinking over the next 3-5 years, then AI is a novelty in most cases, a useful tool in some cases, and so

What kind of time frame is "any time soon"? If you're thinking over the next 3-5 years, then AI is a novelty in most cases, a useful tool in some cases, and something truly disruptive in a fairly narrow set of cases. If you expand the time frame to 10-20 years, then AI is poised to become something that truly upsets most of the economy. Obviously, nothing can be said for certain, there could be black swan events, etc. But on the current trajectory of cheaper, more powerful compute and better understanding of the engineering around getting useful results out of AI, you'd be foolish to think any white collar profession won't be deeply affected by the technological shift that's coming.

ch4s3 2y ago
> If you expand the time frame to 10-20 years This feels like a lot of "20 years away" type predictions. There needs to be some sort of step change in the technology, and perhaps more than one or on more than one front to make this possible but its far enough away that 20 years feels like a safe bet. When people say this about LLM based AIs, I wonder how they imagine text prediction making those step changes. Is your thinking that text prediction is generalization enough that doing more of it better, faster, and on scaled hardware is sufficient to hit a tipping point?
- nostrademons 2y ago
  I think the idea behind the "10-20 year" timeframe is that that's how long current technological changes will take to propagate into social & economic changes, like how HTTP was invented in 1989 but we didn't get e-commerce until 1995 or social networking until 2002. LLMs dramatically reduce the price of bullshit. It follows that they'll likely reduce the value of bullshit too, but we don't know to what extent or how that'll affect the organization of the economy until entrepreneurs generate vast bullshit factories and counterparties start adjusting their behavior to adapt to it. A huge swath of the economy runs on bullshit, from media to PR to sales to venture capital to politics. How these industries are structured will likely change when writing vast quantities of truthy-enough-to-be-convincing text becomes insanely cheap.
randomdata 2y ago
What is "AI"? Definitions vary, but all definitions I have heard seem to boil down to one thing: AI is little more than a set of fancy programming languages. Which means you still always need a human to be the programmer. Nothing really can change for the white collar so long as you still have to provide instruction. That's what white collar people do. "AGI" draws a distinction – where the machines start to be able to act as completely independent agents without needing a human to give instruction. Is this, perhaps, what you are envisioning in 10-20 years?
therobots927 2y ago
There are a lot of assumptions built into your statement that AI will “upset” most of the economy in 20 years. Impact? Sure. But your argument seems to be implying AGI will be achieved (at scale, no less) in 20 years. This is an extraordinary claim even if Altman likes to throw out 5 year timelines for AGI. Similar to how musk (who is also a VC not an engineer or scientist) has been saying self driving teslas were right around the corner for the last ~7 years. AI today is not even close to replicating the brain’s ability to perform multiple tasks together without suddenly hallucinating a totally different task at multiple points in the project. LLMs are good at regurgitating text, I would hope so after they just memorized all available text data. I can feel the chill of the next AI winter right around the corner, personally.
lolinder 2y ago
> If you expand the time frame to 10-20 years, then AI is poised to become something that truly upsets most of the economy. Obviously, nothing can be said for certain, there could be black swan events, etc. But on the current trajectory [...] This is an interesting framing that I think exposes a key difference between people who view AI dominance as inevitable and those who don't. You view the progress of AI slowing down as a black swan event, something that's extremely unlikely. I, on the other hand, view it as inevitable. Technology has never progressed along a single axis linearly or exponentially for very long. Technology progresses in bursts—there's a major breakthrough that leads to a huge amount of interest and progress which leads to rapid growth in a particular area until it saturates. After a few years we start to see diminishing returns as the area opened by the breakthrough becomes fully explored, and eventually progress becomes very slow indeed until the next major breakthrough. Given this general pattern of progress in technology, it would be very foolish indeed to predict the next 20 years' trajectory of AI less than 2 years after ChatGPT was released.
- therobots927 2y ago
  You worded your comment much better than I did, and you touch on the key point in all of this which is the assumption that chatGPT etc will continue to get noticeably better at regular intervals. It’s entirely possible that openAI has reached a local maximum with the data, compute, and model structure they have available. The data is tapped out for the most part. They already probably acquired 80% of the available text training data. Compute is now heavily cost constrained. This is obvious because even GPTs responses have gotten worse over the last 6 months, especially with image generation tasks. Likely because they’re throttling compute. Model structure: OpenAI has admitted that they cannot understand why chatGPT has generated a given output. If you can’t explain your model you can’t improve it either. And more compute won’t help you that much here if I had to guess.
- HarHarVeryFunny 2y ago
  I also think human level (and beyond) AGI is inevitable, but that in of itself says nothing about the time frame in which it will be achieved. People have been predicting AI/AGI ever since the advent of computers over 70 years ago, and estimates have always been way over-optimistic. It's perhaps relevant to note that the reason estimates have always been off seems to be that people have always thought that the technology of the day was sufficient, and that it was just a matter of applying it. So far we've had procedural programming (SHRLDU), expert systems (CYC), general-purpose problem solvers (SOAR), PDP/connectionist approaches, modern neural nets, LLMs... I do think we're getting closer, and that neural nets will get us there (maybe not with gradient descent as the learning method though). There have been lots of valuable lessons and insights learned from various neural net architecture, not least of which is the confirmation by LLMs that prediction seems to be the basis (or at least one of major pillars) of intelligence. And yet, I can't help but feel that there is still a long way to go ...
HarHarVeryFunny 2y ago
> What kind of time frame is "any time soon" I'm guessing that solving run-time learning may well require a different approach, and it's not clear that reasoning (in general form - ability to dynamically synthesize a problem-specific solution) can be just added to LLMs either (e.g. by adding tree search). There are also other missing components such as working memory that seem simpler to solve. Coming up with brand new architectures and learning approaches is likely to take time. There have been attempts to find alternatives to gradient descent, but none very successful despite a lot of effort. Perhaps it's just a reflection of people chasing the low hanging fruit (and as Chollet says, LLMs "sucking all the oxygen out of the room"), but architectural advance post-LLM has been minimal. In 7 years we've basically just gone from transformer paper to big pre-trained transformers. Even when workable architectural approaches to run-time learning and reasoning have been developed, they will also need to be scaled up (another 7 years?), and will also be competing with LLMs for mindshare and dev. resources as long as scaling LLMs continues to be seen as profitable. The timescale for coming up with new architectures and approaches is hard to predict. AGI prediction timeframes have always been wrong, and the transformer was really one of history's accidental discoveries. Who'd have guessed that a better seq-to-seq model would create such capabilities! If I had to guess, I'd say human-level AGI (human-level in terms of both capability and generality) is still 15-20 years away at least. 7 years to go from small transformers to big transformers doesn't make me optimistic that architectural innovation is going to happen very quickly, and anyways this is an unpredictable research problem, not an engineering one.
- ryandvm 2y ago
  But even now the LLMs absolutely have limited problem solving capability. For example, yesterday I asked GPT-4o to write multiple alternate endings to the short story "The Last Equation". They weren't dramatically compelling, but they were logical and functional. How is that not problem solving? And so help me, before anyone tells me it's just stringing together the next most likely tokens - I don't care. Clearly that is at least a primitive form of intelligence. Actually it's not even apparent to me that that isn't exactly what human intelligence is doing...
  - 2y ago

ch4s3 2y ago

> If you expand the time frame to 10-20 years This feels like a lot of "20 years away" type predictions. There needs to be some sort of step change in the technology, and perhaps more than one or on more than one front to make this possible but its far enough away that 20 years feels like a safe bet. When people say this about LLM based AIs, I wonder how they imagine text prediction making those step changes. Is your thinking that text prediction is generalization enough that doing more of it better, faster, and on scaled hardware is sufficient to hit a tipping point?

nostrademons 2y ago
I think the idea behind the "10-20 year" timeframe is that that's how long current technological changes will take to propagate into social & economic changes, like how HTTP was invented in 1989 but we didn't get e-commerce until 1995 or social networking until 2002. LLMs dramatically reduce the price of bullshit. It follows that they'll likely reduce the value of bullshit too, but we don't know to what extent or how that'll affect the organization of the economy until entrepreneurs generate vast bullshit factories and counterparties start adjusting their behavior to adapt to it. A huge swath of the economy runs on bullshit, from media to PR to sales to venture capital to politics. How these industries are structured will likely change when writing vast quantities of truthy-enough-to-be-convincing text becomes insanely cheap.

randomdata 2y ago

What is "AI"? Definitions vary, but all definitions I have heard seem to boil down to one thing: AI is little more than a set of fancy programming languages. Which means you still always need a human to be the programmer. Nothing really can change for the white collar so long as you still have to provide instruction. That's what white collar people do. "AGI" draws a distinction – where the machines start to be able to act as completely independent agents without needing a human to give instruction. Is this, perhaps, what you are envisioning in 10-20 years?

therobots927 2y ago

There are a lot of assumptions built into your statement that AI will “upset” most of the economy in 20 years. Impact? Sure. But your argument seems to be implying AGI will be achieved (at scale, no less) in 20 years. This is an extraordinary claim even if Altman likes to throw out 5 year timelines for AGI. Similar to how musk (who is also a VC not an engineer or scientist) has been saying self driving teslas were right around the corner for the last ~7 years. AI today is not even close to replicating the brain’s ability to perform multiple tasks together without suddenly hallucinating a totally different task at multiple points in the project. LLMs are good at regurgitating text, I would hope so after they just memorized all available text data. I can feel the chill of the next AI winter right around the corner, personally.

lolinder 2y ago

> If you expand the time frame to 10-20 years, then AI is poised to become something that truly upsets most of the economy. Obviously, nothing can be said for certain, there could be black swan events, etc. But on the current trajectory [...] This is an interesting framing that I think exposes a key difference between people who view AI dominance as inevitable and those who don't. You view the progress of AI slowing down as a black swan event, something that's extremely unlikely. I, on the other hand, view it as inevitable. Technology has never progressed along a single axis linearly or exponentially for very long. Technology progresses in bursts—there's a major breakthrough that leads to a huge amount of interest and progress which leads to rapid growth in a particular area until it saturates. After a few years we start to see diminishing returns as the area opened by the breakthrough becomes fully explored, and eventually progress becomes very slow indeed until the next major breakthrough. Given this general pattern of progress in technology, it would be very foolish indeed to predict the next 20 years' trajectory of AI less than 2 years after ChatGPT was released.

therobots927 2y ago
You worded your comment much better than I did, and you touch on the key point in all of this which is the assumption that chatGPT etc will continue to get noticeably better at regular intervals. It’s entirely possible that openAI has reached a local maximum with the data, compute, and model structure they have available. The data is tapped out for the most part. They already probably acquired 80% of the available text training data. Compute is now heavily cost constrained. This is obvious because even GPTs responses have gotten worse over the last 6 months, especially with image generation tasks. Likely because they’re throttling compute. Model structure: OpenAI has admitted that they cannot understand why chatGPT has generated a given output. If you can’t explain your model you can’t improve it either. And more compute won’t help you that much here if I had to guess.
HarHarVeryFunny 2y ago
I also think human level (and beyond) AGI is inevitable, but that in of itself says nothing about the time frame in which it will be achieved. People have been predicting AI/AGI ever since the advent of computers over 70 years ago, and estimates have always been way over-optimistic. It's perhaps relevant to note that the reason estimates have always been off seems to be that people have always thought that the technology of the day was sufficient, and that it was just a matter of applying it. So far we've had procedural programming (SHRLDU), expert systems (CYC), general-purpose problem solvers (SOAR), PDP/connectionist approaches, modern neural nets, LLMs... I do think we're getting closer, and that neural nets will get us there (maybe not with gradient descent as the learning method though). There have been lots of valuable lessons and insights learned from various neural net architecture, not least of which is the confirmation by LLMs that prediction seems to be the basis (or at least one of major pillars) of intelligence. And yet, I can't help but feel that there is still a long way to go ...

HarHarVeryFunny 2y ago

> What kind of time frame is "any time soon" I'm guessing that solving run-time learning may well require a different approach, and it's not clear that reasoning (in general form - ability to dynamically synthesize a problem-specific solution) can be just added to LLMs either (e.g. by adding tree search). There are also other missing components such as working memory that seem simpler to solve. Coming up with brand new architectures and learning approaches is likely to take time. There have been attempts to find alternatives to gradient descent, but none very successful despite a lot of effort. Perhaps it's just a reflection of people chasing the low hanging fruit (and as Chollet says, LLMs "sucking all the oxygen out of the room"), but architectural advance post-LLM has been minimal. In 7 years we've basically just gone from transformer paper to big pre-trained transformers. Even when workable architectural approaches to run-time learning and reasoning have been developed, they will also need to be scaled up (another 7 years?), and will also be competing with LLMs for mindshare and dev. resources as long as scaling LLMs continues to be seen as profitable. The timescale for coming up with new architectures and approaches is hard to predict. AGI prediction timeframes have always been wrong, and the transformer was really one of history's accidental discoveries. Who'd have guessed that a better seq-to-seq model would create such capabilities! If I had to guess, I'd say human-level AGI (human-level in terms of both capability and generality) is still 15-20 years away at least. 7 years to go from small transformers to big transformers doesn't make me optimistic that architectural innovation is going to happen very quickly, and anyways this is an unpredictable research problem, not an engineering one.

ryandvm 2y ago
But even now the LLMs absolutely have limited problem solving capability. For example, yesterday I asked GPT-4o to write multiple alternate endings to the short story "The Last Equation". They weren't dramatically compelling, but they were logical and functional. How is that not problem solving? And so help me, before anyone tells me it's just stringing together the next most likely tokens - I don't care. Clearly that is at least a primitive form of intelligence. Actually it's not even apparent to me that that isn't exactly what human intelligence is doing...
- 2y ago