4 ms·

Look where you were 3 years ago, and where you are now. And then imagine where you will be in 5 more years. If it can almost get a complex problem right now,

by spaceman_2020 2y ago

Look where you were 3 years ago, and where you are now. And then imagine where you will be in 5 more years. If it can almost get a complex problem right now, I'm dead sure it will get it correct within 5 years

colonelspace 2y ago
> I'm dead sure it will get it correct within 5 years You might be right. But plenty of people said we'd all be getting around in self-driving cars for sure 10 years ago.
- neevans 2y ago
  we do have self driving car but since it directly affects people's life it needs to be close to 100% accurate and no margin of errors. Not necessarily the case for LLMs.
  - jmb99 2y ago
    No, we have cars that can drive themselves quite well in good weather, but fail completely in heavy snow/poor visibility. Which is actually a great analogy to LLMs - they work great in the simple cases (80% of the time), it’s that last 20% that’s substantially harder.
AnIrishDuck 2y ago
I'm not? The history of AI development is littered with examples of false starts, hidden traps, and promising breakthroughs that eventually expose deeper and more difficult problems [1]. I wouldn't be shocked if it could eventually get it right, but dead sure? 1. https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter
taco_emoji 2y ago
Have you never heard of "local maxima"? Why are you so certain another 5 years will provide any qualitative advancement at all?
mogoh 2y ago
But can it now say "I don't know." ? Or can it evaluate its own results and came to the conclusion that its just a wild guess? I am still impressed by the progress though.
evilfred 2y ago
what makes you so "dead sure"? it's just hallucinating as always
methodical 2y ago
You're dead sure? I wouldn't say anything definite about technology advancements. People seem to underestimate the last 20% of the problem and only focus on the massive 80% improvements up to this point.
zer0tonin 2y ago
The progress since GPT-3 hasn't been spectacularly fast.
ActorNightly 2y ago
Getting complex problem = having the solution in some form in the training dataset. All we are gonna get is better and better googles.
- olalonde 2y ago
  Why?
  - ActorNightly 2y ago
    Lets say that you want to make a flying car, that can also double as a submarine. Nobody has done this yet. So information doesn't exist on how to do it. An LLM may give you some generic answers from training sets on what engineering/analysis tasks to do, but it won't be able to give you a complex and complete design for one. A model that can actually solve problems would be able to design you one.
    - olalonde 2y ago
      They can solve problems that are not in their training set. There are many examples in the article...
      - ActorNightly 2y ago
        I literally just gave you an example of one it can't solve, despite having a vast knowledge of mechanical and aeronautical subjects. All the examples are obviously in its training set. Here is another better example - none of these models can create a better ML accelerator despite having a wide array of electrical and computer engineering knowledge. If they did, OpenAI would pretty much be printing their own chips like Google does.
        olalonde 2y ago
        In your previous comment you stated that LLMs can only solve problems that are in their training set (e.g. "all we are gonna get is better and better googles"). But that's not true as I pointed out. Now your argument seems to be that they can't solve all problems or, more charitably, can't solve highly complex problems. This is true but by that standard, the vast majority of humans can't reason either. Yes, the reasoning capacities of current LLMs are limited but it's incorrect to pretend they can't reason at all.
closeparen 2y ago
It is not at all clear that "produce correct answer" is the natural endpoint of "produce plausible on-topic utterances that look like they could be answers." To do the former you need to know something about the underlying structure of reality (or have seen the answer before), to do the latter you only need to be good at pattern-matching and language.
dontlikeyoueith 2y ago
I still don't have a Mr. Fusion in my house, FYI. We always overestimate the future.
canjobear 2y ago
Going back 3 years, it feels like incredible progress. Going back 1 year, it feels like pretty much the same limitations.

colonelspace 2y ago

> I'm dead sure it will get it correct within 5 years You might be right. But plenty of people said we'd all be getting around in self-driving cars for sure 10 years ago.

neevans 2y ago
we do have self driving car but since it directly affects people's life it needs to be close to 100% accurate and no margin of errors. Not necessarily the case for LLMs.
- jmb99 2y ago
  No, we have cars that can drive themselves quite well in good weather, but fail completely in heavy snow/poor visibility. Which is actually a great analogy to LLMs - they work great in the simple cases (80% of the time), it’s that last 20% that’s substantially harder.

AnIrishDuck 2y ago

I'm not? The history of AI development is littered with examples of false starts, hidden traps, and promising breakthroughs that eventually expose deeper and more difficult problems [1]. I wouldn't be shocked if it could eventually get it right, but dead sure? 1. https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter

taco_emoji 2y ago

Have you never heard of "local maxima"? Why are you so certain another 5 years will provide any qualitative advancement at all?

mogoh 2y ago

But can it now say "I don't know." ? Or can it evaluate its own results and came to the conclusion that its just a wild guess? I am still impressed by the progress though.

evilfred 2y ago

what makes you so "dead sure"? it's just hallucinating as always

methodical 2y ago

You're dead sure? I wouldn't say anything definite about technology advancements. People seem to underestimate the last 20% of the problem and only focus on the massive 80% improvements up to this point.

zer0tonin 2y ago

The progress since GPT-3 hasn't been spectacularly fast.

ActorNightly 2y ago

Getting complex problem = having the solution in some form in the training dataset. All we are gonna get is better and better googles.

olalonde 2y ago
Why?
- ActorNightly 2y ago
  Lets say that you want to make a flying car, that can also double as a submarine. Nobody has done this yet. So information doesn't exist on how to do it. An LLM may give you some generic answers from training sets on what engineering/analysis tasks to do, but it won't be able to give you a complex and complete design for one. A model that can actually solve problems would be able to design you one.
  - olalonde 2y ago
    They can solve problems that are not in their training set. There are many examples in the article...
    - ActorNightly 2y ago
      I literally just gave you an example of one it can't solve, despite having a vast knowledge of mechanical and aeronautical subjects. All the examples are obviously in its training set. Here is another better example - none of these models can create a better ML accelerator despite having a wide array of electrical and computer engineering knowledge. If they did, OpenAI would pretty much be printing their own chips like Google does.
      - olalonde 2y ago
        In your previous comment you stated that LLMs can only solve problems that are in their training set (e.g. "all we are gonna get is better and better googles"). But that's not true as I pointed out. Now your argument seems to be that they can't solve all problems or, more charitably, can't solve highly complex problems. This is true but by that standard, the vast majority of humans can't reason either. Yes, the reasoning capacities of current LLMs are limited but it's incorrect to pretend they can't reason at all.

closeparen 2y ago

It is not at all clear that "produce correct answer" is the natural endpoint of "produce plausible on-topic utterances that look like they could be answers." To do the former you need to know something about the underlying structure of reality (or have seen the answer before), to do the latter you only need to be good at pattern-matching and language.

dontlikeyoueith 2y ago

I still don't have a Mr. Fusion in my house, FYI. We always overestimate the future.

canjobear 2y ago

Going back 3 years, it feels like incredible progress. Going back 1 year, it feels like pretty much the same limitations.