4 ms·

Ugh, WSJ - no, it's not "shocking experts". It's shocking people who have absolutely no conception of how GPT-3 works. Those of us who do are mildly impressed,

Ugh, WSJ - no, it's not "shocking experts". It's shocking people who have absolutely no conception of how GPT-3 works. Those of us who do are mildly impressed, but recognize that GPT-3 is only an incremental improvement, and much of what comes out of it is nothing more than moderately coherent nonsense.

red75prime 6y ago
Next step: integrate language model and reinforcement learning. https://arxiv.org/abs/2009.01719 https://arxiv.org/abs/2009.01719
dwaltrip 6y ago
Meh, I think you are over-correcting a bit here. GPT-3 is not just your standard annual incremental progress in AI.
- king_magic 6y ago
  I don't think I am. It is an incremental improvement, albeit a fairly impressive one, and yes, it can do some interesting "mildly new-ish" things, but at the end of the day, it still very often just a nonsense factory. IMO, GANs were a much bigger and far more important step forward than GPT-2 ---> GPT-3.
jonplackett 6y ago
You’re right that WSJ is overdoing it. But you’re definitely underdoing it.
- king_magic 6y ago
  I don't think I'm underdoing it at all. I'm assigning a reasonable level of impressiveness to it. It's cool, it's got a lot of parameters, and it's flexible in very interesting ways. But... am I "shocked" when I look at the output? Not even close. It's about what I would have expected if I threw a few million dollars at training a 175 billion parameter model. Edit: after reading a few more comments, I concede that I may be ever so slightly underdoing it :)
evolve2k 6y ago
The “shock” is not about the difference between GPT-3 against developments on its predecessors. It’s that itself it’s starting to cross the “Writing Barrier”. The incremental gain is of little public interest, the social impacts of having an AI that can reliably produce content that is often indistinguishable from human content, now we’re potentially facing a profound change in how we write and in who’s writing we consume.
- king_magic 6y ago
  This is a fair & interesting perspective - I'll admit, I hadn't considered the "shock" factor in this way.
- EarthLaunch 6y ago
  > who’s writing we consume Maybe it’ll be an improvement
  - evolve2k 6y ago
    Smart writing tools + With regards to myself having a Smart Grammarly tool where I can write a page of dot pints and provide a few web links and then run a browser extension that turns it all into a coherent logical article. That would be brilliant. Even more fake content - The more SEO optimised, computer generate, algorithm ready content that is produced the faster we head into control-based dystopian context. You spend a month and write up your best distilled, researched, coherent new thoughts on where we need to shift as a society. Other parties jam together 1000 response articles drawing on established thought with the intent to deceive and your message is lost in the noise. “It’s all fake anyways” An acceleration of our information downfall.

red75prime 6y ago

Next step: integrate language model and reinforcement learning. https://arxiv.org/abs/2009.01719 https://arxiv.org/abs/2009.01719

dwaltrip 6y ago

Meh, I think you are over-correcting a bit here. GPT-3 is not just your standard annual incremental progress in AI.

king_magic 6y ago
I don't think I am. It is an incremental improvement, albeit a fairly impressive one, and yes, it can do some interesting "mildly new-ish" things, but at the end of the day, it still very often just a nonsense factory. IMO, GANs were a much bigger and far more important step forward than GPT-2 ---> GPT-3.

jonplackett 6y ago

You’re right that WSJ is overdoing it. But you’re definitely underdoing it.

king_magic 6y ago
I don't think I'm underdoing it at all. I'm assigning a reasonable level of impressiveness to it. It's cool, it's got a lot of parameters, and it's flexible in very interesting ways. But... am I "shocked" when I look at the output? Not even close. It's about what I would have expected if I threw a few million dollars at training a 175 billion parameter model. Edit: after reading a few more comments, I concede that I may be ever so slightly underdoing it :)

evolve2k 6y ago

The “shock” is not about the difference between GPT-3 against developments on its predecessors. It’s that itself it’s starting to cross the “Writing Barrier”. The incremental gain is of little public interest, the social impacts of having an AI that can reliably produce content that is often indistinguishable from human content, now we’re potentially facing a profound change in how we write and in who’s writing we consume.

king_magic 6y ago
This is a fair & interesting perspective - I'll admit, I hadn't considered the "shock" factor in this way.
EarthLaunch 6y ago
> who’s writing we consume Maybe it’ll be an improvement
- evolve2k 6y ago
  Smart writing tools + With regards to myself having a Smart Grammarly tool where I can write a page of dot pints and provide a few web links and then run a browser extension that turns it all into a coherent logical article. That would be brilliant. Even more fake content - The more SEO optimised, computer generate, algorithm ready content that is produced the faster we head into control-based dystopian context. You spend a month and write up your best distilled, researched, coherent new thoughts on where we need to shift as a society. Other parties jam together 1000 response articles drawing on established thought with the intent to deceive and your message is lost in the noise. “It’s all fake anyways” An acceleration of our information downfall.