3 ms·

What emergent capabilities exactly? I’m not sure “no one predicted this” is accurate, more like there was a lot of hype in the research community circa 2018 whe

What emergent capabilities exactly? I’m not sure “no one predicted this” is accurate, more like there was a lot of hype in the research community circa 2018 when this stuff developed, and it’s just not news these days research-wise

cubefox 3y ago
It absolutely was accurate. Back in 2018 (GPT-1) nobody expected we would soon have something as powerful as GPT-3, or even ChatGPT. The massive progress in LLMs (and text-to-image models) took everyone by surprise. They achieved things that, until recently, were regarded as pure science fiction about the distant future.
- bamboozled 3y ago
  What’s so “powerful” about ChatGPT 3 and 4 ?
  - ruszki 3y ago
    They respond so well that a lot of people believe their results without question, even while we know that they hallucinate all the time. They’re good enough for many. Even here in HN, where everybody should know that they are terrible with numbers, majority of us cheered that somebody generated diagrams with them, I couldn’t even find a single comment which were about “wait a minute, they’re fucking unreliable numbers”. Also regarding coding, they can definitely produce junior level code, especially if you follow a similar trial and error way of thinking which happens all the time during your work, if you try to use an unknown interface.
  - lambdaone 3y ago
    Their ability to superficially mimic cognition, in a way that not only raises significant philosophical questions but also seems (rightly or wrongly) to bring the prospect of "true" AI tantalizingly closer.
    - bamboozled 3y ago
      What are the new philosophical questions that it raises for you ? Existential crisis ?
    - kaba0 3y ago
      > bring the prospect of "true" AI tantalizingly closer I do think that it’s quite a leap to assume that. It is very good at mimicking, but the same way some parrots can talk, they are endlessly far from human cognition. That jump is very very non-trivial.
  - MrDisposable 3y ago
    A stochastic parrot, a blurry jpeg of the web is able to provide sound, actionable, verifiable advice on topics such as modeling in Blender or managing a Linux server -- despite not being generally intelligent.
  - cubefox 3y ago
    [flagged]
    - bamboozled 3y ago
      No. Powerful implies the thing is The Emperor from Star Wars or something. Capable in certain circumstances ? Yes.
- PeterisP 3y ago
  I feel that "soon" is doing a lot of the work here, as far as I recall back in 2018 people in the research community absolutely were expecting something similar to GPT-3, only the expectation was that these capabilities would land perhaps 2-3 years later than they did. And at least part of that was the (implied, natural, understandable) "linear extrapolation" of the total cost of compute applied - if your intuitive expectation of progress if effectively "oh, what would happen next year if we'd triple the budget" but actually the applied compute power increases hundredfold, then obviously the outcome arrives faster than you expected.
LikesPwsh 3y ago
Getting good enough that it's impossible to tell whether it contains a particular copyrighted (or GPL) work. That's made them effectively immune to copyright enforcement so far; I'd go as far as saying the main function of llm/diffusion models is obfuscation of copyright breaches.

cubefox 3y ago

It absolutely was accurate. Back in 2018 (GPT-1) nobody expected we would soon have something as powerful as GPT-3, or even ChatGPT. The massive progress in LLMs (and text-to-image models) took everyone by surprise. They achieved things that, until recently, were regarded as pure science fiction about the distant future.

bamboozled 3y ago
What’s so “powerful” about ChatGPT 3 and 4 ?
- ruszki 3y ago
  They respond so well that a lot of people believe their results without question, even while we know that they hallucinate all the time. They’re good enough for many. Even here in HN, where everybody should know that they are terrible with numbers, majority of us cheered that somebody generated diagrams with them, I couldn’t even find a single comment which were about “wait a minute, they’re fucking unreliable numbers”. Also regarding coding, they can definitely produce junior level code, especially if you follow a similar trial and error way of thinking which happens all the time during your work, if you try to use an unknown interface.
- lambdaone 3y ago
  Their ability to superficially mimic cognition, in a way that not only raises significant philosophical questions but also seems (rightly or wrongly) to bring the prospect of "true" AI tantalizingly closer.
  - bamboozled 3y ago
    What are the new philosophical questions that it raises for you ? Existential crisis ?
  - kaba0 3y ago
    > bring the prospect of "true" AI tantalizingly closer I do think that it’s quite a leap to assume that. It is very good at mimicking, but the same way some parrots can talk, they are endlessly far from human cognition. That jump is very very non-trivial.
- MrDisposable 3y ago
  A stochastic parrot, a blurry jpeg of the web is able to provide sound, actionable, verifiable advice on topics such as modeling in Blender or managing a Linux server -- despite not being generally intelligent.
- cubefox 3y ago
  [flagged]
  - bamboozled 3y ago
    No. Powerful implies the thing is The Emperor from Star Wars or something. Capable in certain circumstances ? Yes.
PeterisP 3y ago
I feel that "soon" is doing a lot of the work here, as far as I recall back in 2018 people in the research community absolutely were expecting something similar to GPT-3, only the expectation was that these capabilities would land perhaps 2-3 years later than they did. And at least part of that was the (implied, natural, understandable) "linear extrapolation" of the total cost of compute applied - if your intuitive expectation of progress if effectively "oh, what would happen next year if we'd triple the budget" but actually the applied compute power increases hundredfold, then obviously the outcome arrives faster than you expected.

LikesPwsh 3y ago

Getting good enough that it's impossible to tell whether it contains a particular copyrighted (or GPL) work. That's made them effectively immune to copyright enforcement so far; I'd go as far as saying the main function of llm/diffusion models is obfuscation of copyright breaches.