4 ms·

I wonder if that could be a viable approach, having a separate "writer" and "editor" AI

jcims 5y ago
I do the same thing when I write (except for my comments on HN lol) GPT-3 reads like the unfiltered garbage that comes out of my head when I'm trying to get ideas out. The primary difference is that I have empathy for the audience so I go through the effort to reread, clean it up, clear it up then cut it down. But that secondary editing process has nothing to do with the subject matter or initial content generation process. It was trained through years of *not* doing it and having my audience ask lots of questions and clearly misunderstand what I was trying to say (let's call it reinforcement learning). As a result I now have an intuition about which concepts need explanatory analogies, based on my understanding of what the readers know, and which are almost certainly going to be understood outright. I think this gets at the root of the difference. Basically when I write, I'm not trying to generate text, I'm trying to generate training data that will amend foreign models with a high fidelity representation of a subset of mine. This feels GAN-like but different. It's like you need to have two separate models, the 'expert' and the 'n00b' (or ten n00bs trained on different data), ask both of them the same thing, and one-shot retrain the n00b from the answers provided by the 'expert' until the n00b is able to generate content that the expert finds to be sufficiently similar.
- isaacimagine 5y ago
  > Basically when I write, I'm not trying to generate text, I'm trying to generate training data that will amend foreign models with a high fidelity representation of a subset of mine. Well said! Replying so I can find this thread again, please ignore.
  - btrettel 5y ago
    In the future, rather than cluttering up the thread, bookmark the page in your browser or use the "favorite" button here on HN.
syrrim 5y ago
GPT-3 is the editor. The hard part of writing is getting a decent idea onto the page. GPT-3 can't do that, it doesn't think for itself. It is good at the second part of writing, creating a coherent string of sentences describing an idea. But since GPT-3 doesn't have decent ideas, the final product doesn't end up being interesting either. The most interesting use cases for GPT-3 involve pairing it with a human who will contribute the actual thinking to the project, with GPT being there to come up with interesting turns of phrase and nice conjoining sentences. Having an AI in the driver seat is still a long way off.

I do the same thing when I write (except for my comments on HN lol) GPT-3 reads like the unfiltered garbage that comes out of my head when I'm trying to get ideas out. The primary difference is that I have empathy for the audience so I go through the effort to reread, clean it up, clear it up then cut it down. But that secondary editing process has nothing to do with the subject matter or initial content generation process. It was trained through years of *not* doing it and having my audience ask lots of questions and clearly misunderstand what I was trying to say (let's call it reinforcement learning). As a result I now have an intuition about which concepts need explanatory analogies, based on my understanding of what the readers know, and which are almost certainly going to be understood outright. I think this gets at the root of the difference. Basically when I write, I'm not trying to generate text, I'm trying to generate training data that will amend foreign models with a high fidelity representation of a subset of mine. This feels GAN-like but different. It's like you need to have two separate models, the 'expert' and the 'n00b' (or ten n00bs trained on different data), ask both of them the same thing, and one-shot retrain the n00b from the answers provided by the 'expert' until the n00b is able to generate content that the expert finds to be sufficiently similar.

isaacimagine 5y ago
> Basically when I write, I'm not trying to generate text, I'm trying to generate training data that will amend foreign models with a high fidelity representation of a subset of mine. Well said! Replying so I can find this thread again, please ignore.
- btrettel 5y ago
  In the future, rather than cluttering up the thread, bookmark the page in your browser or use the "favorite" button here on HN.

syrrim 5y ago

GPT-3 is the editor. The hard part of writing is getting a decent idea onto the page. GPT-3 can't do that, it doesn't think for itself. It is good at the second part of writing, creating a coherent string of sentences describing an idea. But since GPT-3 doesn't have decent ideas, the final product doesn't end up being interesting either. The most interesting use cases for GPT-3 involve pairing it with a human who will contribute the actual thinking to the project, with GPT being there to come up with interesting turns of phrase and nice conjoining sentences. Having an AI in the driver seat is still a long way off.