3 ms·

Eh, it's still impressive that these systems can write such good code despite pretty much just predicting the next word. I guess it's a matter of perspective.

Eh, it's still impressive that these systems can write such good code despite pretty much just predicting the next word. I guess it's a matter of perspective. You can either be astounded by how much it can do relative to your expectations from 2018 or you can be skeptical relative to the volume of excitement. I think some excitement is also people extrapolating to the future: if predicting the next word gets you this far, what happens when you actually try to make it good?

copperx 4y ago
> despite pretty much just predicting the next word Is GPT really like the old school NLP trigram stuff?
- hnfong 4y ago
  "predicting the next word" sounds trivial until you realize the value and complexity of "predicting the next word[s] by a world leading expert in a particular domain".
  - Jensson 4y ago
    Quoting famous people sounds smart until you realize they just memorized a ton of trivia. These models have demonstrated that they don't learn logical models, instead they learn to generate text that looks logical at first glance but is nonsense.
    - sgarland 4y ago
      I asked GPT-4 something fairly niche that I happen to know a fair amount about: to explain the concept of Xenon poisoning in a nuclear reactor. Other than skipping Te-135 being the initial fission product that starts the decay chain (and tbf, operationally it can be skipped since the half-life is 19 seconds), it got everything correct. I'm sure if I kept probing on smaller and smaller details it would eventually fail, but I'd argue for _most_ people, on _most_ subjects, it performs incredibly well.
      - eternalban 4y ago
        I couldn't get ChatGPT to do Wordle. I don't have access to GPT-4. See if it can do that.
      - hungryforcodes 4y ago
        Ok, but it has all this information already. So why is this surprising. A real test of it's abilities would be synthesizing a NEW type of nuclear reactor.
pms 4y ago
> what happens when you actually try to make it good? 1. Do you think that companies paying millions of dollars to ML researchers aren't already trying to make it good? 2. I think it will take a real revolution in AI/ML to do what people here are extrapolating into the future. That revolution will eventually come, but I doubt it'll be as quick as people think. Just think about the excitement people had about Siri 10+ years ago, or about Full Self Driving 5+ years ago. In my opinion, in 5-10 years from now GPT will be in the same place where Siri and Full Self Driving currently are. Eventually we will make the leap we're dreaming of, but that leap isn't happening yet.

copperx 4y ago

> despite pretty much just predicting the next word Is GPT really like the old school NLP trigram stuff?

hnfong 4y ago
"predicting the next word" sounds trivial until you realize the value and complexity of "predicting the next word[s] by a world leading expert in a particular domain".
- Jensson 4y ago
  Quoting famous people sounds smart until you realize they just memorized a ton of trivia. These models have demonstrated that they don't learn logical models, instead they learn to generate text that looks logical at first glance but is nonsense.
  - sgarland 4y ago
    I asked GPT-4 something fairly niche that I happen to know a fair amount about: to explain the concept of Xenon poisoning in a nuclear reactor. Other than skipping Te-135 being the initial fission product that starts the decay chain (and tbf, operationally it can be skipped since the half-life is 19 seconds), it got everything correct. I'm sure if I kept probing on smaller and smaller details it would eventually fail, but I'd argue for _most_ people, on _most_ subjects, it performs incredibly well.
    - eternalban 4y ago
      I couldn't get ChatGPT to do Wordle. I don't have access to GPT-4. See if it can do that.
    - hungryforcodes 4y ago
      Ok, but it has all this information already. So why is this surprising. A real test of it's abilities would be synthesizing a NEW type of nuclear reactor.

pms 4y ago

> what happens when you actually try to make it good? 1. Do you think that companies paying millions of dollars to ML researchers aren't already trying to make it good? 2. I think it will take a real revolution in AI/ML to do what people here are extrapolating into the future. That revolution will eventually come, but I doubt it'll be as quick as people think. Just think about the excitement people had about Siri 10+ years ago, or about Full Self Driving 5+ years ago. In my opinion, in 5-10 years from now GPT will be in the same place where Siri and Full Self Driving currently are. Eventually we will make the leap we're dreaming of, but that leap isn't happening yet.