3 ms·

It's a nice touch that the author defends humans, but just like the ones telling that playing Go is too complicated for machines to be learned, I'm pretty convi

It's a nice touch that the author defends humans, but just like the ones telling that playing Go is too complicated for machines to be learned, I'm pretty convinced that the author is ultimately wrong. I think we need a new approach however. Most of the ML/AI efforts have been directed into the direction of replicating convinving speech at the word and sentence level. I don't think it's possible to reach the goal of novel writing at any stage of extension of this research — simply because that's also not how humans write novels. I think what's missing for GPT-3 et al. (as well as for the models in music that I'm closely following) to reach this point is a new kind of adversarial approach that supplements the current bottom-up methods with a top-down mindset. What we're still missing is the front lobe of deep nets that directs any kind of underlying "high level" syntactic network into the stories and narratives it wants to reach, while supervising setting, pace and form along the way. Once we found that very breakthrough, humanity is toast from there. Just my 2 cents.

commonturtle 6y ago
> simply because that's also not how humans write novels. Planes don't fly by flapping their wings either. It's totally possible we will construct a super capable artificial system that only vaguely resembles the human brain. Personally I think if we managed to create a 10 times more sophisticated GPT-3 (instead of 2048 tokens it looked back 10k+ tokens) we might get pretty close to a novel writing AI.
- redisman 6y ago
  I see it as a multi part process. 1) Generate a rough plot structure. Let's say even just do three acts: motivation -> conflict -> resolution. Now you have a logical throughline and your text generation algorithm doesn't need human level intelligence. This could be randomly generated or some kind of a search from a plot structures database with madlibs inserted into it. Generate 10 characters for the story and insert them into the variables in the plot structure. 2) Use the acts generated above as GPT-3 prompts to fill in the details. If you start iterating on something like that for a while I bet you could get a decent novel out of it that would fool most people. Maybe you need to define each chapter in the first step, maybe it needs more details in the writing prompts, etc.

commonturtle 6y ago

> simply because that's also not how humans write novels. Planes don't fly by flapping their wings either. It's totally possible we will construct a super capable artificial system that only vaguely resembles the human brain. Personally I think if we managed to create a 10 times more sophisticated GPT-3 (instead of 2048 tokens it looked back 10k+ tokens) we might get pretty close to a novel writing AI.

redisman 6y ago
I see it as a multi part process. 1) Generate a rough plot structure. Let's say even just do three acts: motivation -> conflict -> resolution. Now you have a logical throughline and your text generation algorithm doesn't need human level intelligence. This could be randomly generated or some kind of a search from a plot structures database with madlibs inserted into it. Generate 10 characters for the story and insert them into the variables in the plot structure. 2) Use the acts generated above as GPT-3 prompts to fill in the details. If you start iterating on something like that for a while I bet you could get a decent novel out of it that would fool most people. Maybe you need to define each chapter in the first step, maybe it needs more details in the writing prompts, etc.