6 ms·

An AI Breaks the Writing Barrier

41 points by jkuria 6y ago

6gvONxR4sf7o 6y ago
I hate it when models are described as "an AI." It's too much of a personification for my taste. "An intelligence" carries a lot of baggage compared to "a model." We can argue til we're blue in the face about whether GPT-3 is intelligent, so writing a headline like this is already too much hype for me. I've gotten the chance to play with GPT-3 quite a bit, and while amazingly impressive, it's not remotely close to what the hype train says it is.
- king_magic 6y ago
  I've had to (nicely) beat usage of the phrase "an AI" out of my team. It's so inaccurate that it's almost dishonest, to your point of the additional anthropomorphic baggage it carries.
- devindotcom 6y ago
  You're right in a way but the term AI has come to encompass all software of the machine learning type, systems that imitate the process of intelligent thought. It's shorthand at this point, unfortunately inaccurate but ML isn't as well known and every character counts in headlines so longer descriptions are saved for the body. I don't like the way some words are used, and I complain about it too: https://coldewey.cc/2020/06/common-bugbears-of-modern-online-writing/ https://coldewey.cc/2020/06/common-bugbears-of-modern-online... That's our right as fellow users of the language. But unfortunately (such as with "begging the question" or "flammable") we're much too late to make a difference. AI is, I think one of those cases.
  - 6gvONxR4sf7o 6y ago
    That's not the complaint. This is orthogonal to the AI vs ML vs stats debate. It's "an AI/ML/stats model" vs "an AI." "An artificial intelligence" has baggage that's hard to pin down, like it suggests it has a notion of self or sentience or something.
Animats 6y ago
This technology will make a big dent in blithering as an industry. Too much of journalism, and far too much of the web, consists of taking in press releases and speeches and rewriting them. That now looks automatable. I wonder if GPT-4 will be able to take over the romance novel industry.
- libertine 6y ago
  >I wonder if GPT-4 will be able to take over the romance novel industry. More likely carve a niche for romances made by AI.
  - jessaustin 6y ago
    The niche will be a multitude of single-reader "filter bubbles". The AI will figure out what the reader wants, and give it to her/him as intensely as possible. Eventually the content produced for particular readers will only be roughly literate, let alone clever or stylish. Other humans might not even be able to read it, but the marketing bots won't have any trouble. One would like to think that literature in general can survive this hyper-personalization, but optimism is cowardice.
    - Cro_on 6y ago
      >The AI will figure out what the reader wants, and give it to her/him as intensely as possible. Absolutely this. But the final form of this requires some input from the reader – front camera access and retina scanning presumably. It may evolve, without proper curtailing, (unidentifiable ads aside) to be a pretty powerful tool of motivation / pacification / manipulation. >Eventually the content produced for particular readers will only be roughly literate, let alone clever or stylish. Don't know about this, I have slightly more faith in people's ability of appreciation. Though I could be embarrassingly wrong. I feel (a little hyperbolic allowances, if you will) it's in the books for an Infinite Jest type of entertainment to arise, be it in kindle or netflix form. That is, a piece of entertainment so excruciatingly entertaining, that people will literally and completely drop whatever it is they were doing before (or, you know, just miss a week of work for the sake of the most intimate and powerful writing they've ever experienced) and bask in the glories of their own personal god revealing truths of the universe to them.
potiuper 6y ago
Q: Which is heavier, a toaster or a pencil? A: A pencil is heavier than a toaster. Seems legit: https://www.guinnessworldrecords.com/world-records/102759-largest-pencil https://www.guinnessworldrecords.com/world-records/102759-la...
- Rapzid 6y ago
  There's that word again,heavy. Why are things so heavy in the future is there a problem with the Earth's gravitational pull?
netsectoday 6y ago
Nowadays when reading articles I usually have a "did a bot write this" meter running in my head. When the author has no point and is just spewing related words that form sentences I usually click away. I wonder at what point it'll switch to "did a human write this" and I will click away when I realize it's inefficient monkey words.
- sjtindell 6y ago
  Perhaps the polarization and politicization here in the US will bring about your scenario #2 that much faster...in the same way we’ve found some people can trust a therapist AI more than a human one, the same will happen for writers.
- Reedx 6y ago
  Soon we'll have that cognitive overhead on HN threads and there will be a larger volume to parse. Will the comments still be worth reading at that point? [ this comment may or may not be GPT-3 generated ]
  - ccozan 6y ago
    There is a very insightful XKCD comic about this but I can't find it. Please anyone?
    - mandelken 6y ago
      https://xkcd.com/810/ https://xkcd.com/810/
  - espadrine 6y ago
    That is a silly discrepancy. Of course machines are capable of valuable discourse, because humans are biological machines, and humans are capable of valuable discourse. Intelligent contributions are welcome whether they come from silicon-based or carbon-based life.
    - throwawayinfo 6y ago
king_magic 6y ago
Ugh, WSJ - no, it's not "shocking experts". It's shocking people who have absolutely no conception of how GPT-3 works. Those of us who do are mildly impressed, but recognize that GPT-3 is only an incremental improvement, and much of what comes out of it is nothing more than moderately coherent nonsense.
- red75prime 6y ago
  Next step: integrate language model and reinforcement learning. https://arxiv.org/abs/2009.01719 https://arxiv.org/abs/2009.01719
- dwaltrip 6y ago
  Meh, I think you are over-correcting a bit here. GPT-3 is not just your standard annual incremental progress in AI.
  - king_magic 6y ago
    I don't think I am. It is an incremental improvement, albeit a fairly impressive one, and yes, it can do some interesting "mildly new-ish" things, but at the end of the day, it still very often just a nonsense factory. IMO, GANs were a much bigger and far more important step forward than GPT-2 ---> GPT-3.
- jonplackett 6y ago
  You’re right that WSJ is overdoing it. But you’re definitely underdoing it.
  - king_magic 6y ago
    I don't think I'm underdoing it at all. I'm assigning a reasonable level of impressiveness to it. It's cool, it's got a lot of parameters, and it's flexible in very interesting ways. But... am I "shocked" when I look at the output? Not even close. It's about what I would have expected if I threw a few million dollars at training a 175 billion parameter model. Edit: after reading a few more comments, I concede that I may be ever so slightly underdoing it :)

41 points by jkuria 6y ago

I hate it when models are described as "an AI." It's too much of a personification for my taste. "An intelligence" carries a lot of baggage compared to "a model." We can argue til we're blue in the face about whether GPT-3 is intelligent, so writing a headline like this is already too much hype for me. I've gotten the chance to play with GPT-3 quite a bit, and while amazingly impressive, it's not remotely close to what the hype train says it is.

king_magic 6y ago
I've had to (nicely) beat usage of the phrase "an AI" out of my team. It's so inaccurate that it's almost dishonest, to your point of the additional anthropomorphic baggage it carries.
devindotcom 6y ago
You're right in a way but the term AI has come to encompass all software of the machine learning type, systems that imitate the process of intelligent thought. It's shorthand at this point, unfortunately inaccurate but ML isn't as well known and every character counts in headlines so longer descriptions are saved for the body. I don't like the way some words are used, and I complain about it too: https://coldewey.cc/2020/06/common-bugbears-of-modern-online-writing/ https://coldewey.cc/2020/06/common-bugbears-of-modern-online... That's our right as fellow users of the language. But unfortunately (such as with "begging the question" or "flammable") we're much too late to make a difference. AI is, I think one of those cases.
- 6gvONxR4sf7o 6y ago
  That's not the complaint. This is orthogonal to the AI vs ML vs stats debate. It's "an AI/ML/stats model" vs "an AI." "An artificial intelligence" has baggage that's hard to pin down, like it suggests it has a notion of self or sentience or something.

Animats 6y ago

This technology will make a big dent in blithering as an industry. Too much of journalism, and far too much of the web, consists of taking in press releases and speeches and rewriting them. That now looks automatable. I wonder if GPT-4 will be able to take over the romance novel industry.

libertine 6y ago
>I wonder if GPT-4 will be able to take over the romance novel industry. More likely carve a niche for romances made by AI.
- jessaustin 6y ago
  The niche will be a multitude of single-reader "filter bubbles". The AI will figure out what the reader wants, and give it to her/him as intensely as possible. Eventually the content produced for particular readers will only be roughly literate, let alone clever or stylish. Other humans might not even be able to read it, but the marketing bots won't have any trouble. One would like to think that literature in general can survive this hyper-personalization, but optimism is cowardice.
  - Cro_on 6y ago
    >The AI will figure out what the reader wants, and give it to her/him as intensely as possible. Absolutely this. But the final form of this requires some input from the reader – front camera access and retina scanning presumably. It may evolve, without proper curtailing, (unidentifiable ads aside) to be a pretty powerful tool of motivation / pacification / manipulation. >Eventually the content produced for particular readers will only be roughly literate, let alone clever or stylish. Don't know about this, I have slightly more faith in people's ability of appreciation. Though I could be embarrassingly wrong. I feel (a little hyperbolic allowances, if you will) it's in the books for an Infinite Jest type of entertainment to arise, be it in kindle or netflix form. That is, a piece of entertainment so excruciatingly entertaining, that people will literally and completely drop whatever it is they were doing before (or, you know, just miss a week of work for the sake of the most intimate and powerful writing they've ever experienced) and bask in the glories of their own personal god revealing truths of the universe to them.

potiuper 6y ago

Q: Which is heavier, a toaster or a pencil? A: A pencil is heavier than a toaster. Seems legit: https://www.guinnessworldrecords.com/world-records/102759-largest-pencil https://www.guinnessworldrecords.com/world-records/102759-la...

Rapzid 6y ago
There's that word again,heavy. Why are things so heavy in the future is there a problem with the Earth's gravitational pull?

netsectoday 6y ago

Nowadays when reading articles I usually have a "did a bot write this" meter running in my head. When the author has no point and is just spewing related words that form sentences I usually click away. I wonder at what point it'll switch to "did a human write this" and I will click away when I realize it's inefficient monkey words.

sjtindell 6y ago
Perhaps the polarization and politicization here in the US will bring about your scenario #2 that much faster...in the same way we’ve found some people can trust a therapist AI more than a human one, the same will happen for writers.
Reedx 6y ago
Soon we'll have that cognitive overhead on HN threads and there will be a larger volume to parse. Will the comments still be worth reading at that point? [ this comment may or may not be GPT-3 generated ]
- ccozan 6y ago
  There is a very insightful XKCD comic about this but I can't find it. Please anyone?
  - mandelken 6y ago
    https://xkcd.com/810/ https://xkcd.com/810/
- espadrine 6y ago
  That is a silly discrepancy. Of course machines are capable of valuable discourse, because humans are biological machines, and humans are capable of valuable discourse. Intelligent contributions are welcome whether they come from silicon-based or carbon-based life.
  - throwawayinfo 6y ago

king_magic 6y ago

Ugh, WSJ - no, it's not "shocking experts". It's shocking people who have absolutely no conception of how GPT-3 works. Those of us who do are mildly impressed, but recognize that GPT-3 is only an incremental improvement, and much of what comes out of it is nothing more than moderately coherent nonsense.

red75prime 6y ago
Next step: integrate language model and reinforcement learning. https://arxiv.org/abs/2009.01719 https://arxiv.org/abs/2009.01719
dwaltrip 6y ago
Meh, I think you are over-correcting a bit here. GPT-3 is not just your standard annual incremental progress in AI.
- king_magic 6y ago
  I don't think I am. It is an incremental improvement, albeit a fairly impressive one, and yes, it can do some interesting "mildly new-ish" things, but at the end of the day, it still very often just a nonsense factory. IMO, GANs were a much bigger and far more important step forward than GPT-2 ---> GPT-3.
jonplackett 6y ago
You’re right that WSJ is overdoing it. But you’re definitely underdoing it.
- king_magic 6y ago
  I don't think I'm underdoing it at all. I'm assigning a reasonable level of impressiveness to it. It's cool, it's got a lot of parameters, and it's flexible in very interesting ways. But... am I "shocked" when I look at the output? Not even close. It's about what I would have expected if I threw a few million dollars at training a 175 billion parameter model. Edit: after reading a few more comments, I concede that I may be ever so slightly underdoing it :)