6 ms·

Surely we are not far off models capable of submission-quality essays that will enable a new generation of cheating.

by chaz6 7y ago

JRKrause 7y ago
From my observation, even the largest GPT-2 model has difficulty retaining any long-range relationship information. In the "unicorn" writing example that was published originally, the model 'forgets' where the researchers are (climbing a mountain versus being beside a lake iirc) after just a few sentences. Because of this, it's hard to imagine models of this type being able to write long-form coherent papers. Now if we could somehow constrain the generated text to conform to a predefined graph structure that isn't forgotten so quickly...
- jsinai 7y ago
  Maybe the problem is that most of these models seem to rely on sequential information (even the transformer needs this for forward generation of text) to encode long range information. But I can’t remember the last time I relied on sequentially remembering the ordering of tokens in order to complete an essay or hell even reply to an email. Structurally we retain some kind of hierarchical information (topic, places, names, events) about text. Is there any active research looking into text generation models which do this? Maybe some kind of query that is made in a learned vector space and which is not temporarily dependent but rather “spatially” - as in these are the facts about the text being generated so far.
  - flancian 7y ago
    I'm interested on this as well. I have been trying to fine-tune GPT-2 on genre fiction to work as a sort of "fiction replicator". Stylistically it actually seems to do quite reasonably, but it lacks narrative cohesion. This problem, as you point out, is corpus agnostic. I thought of trying to keep track of characters and key interactions outside of the model, but I haven't figured out how to make these two models interact reliably -- outside of just having the first component generate prompts for the second model in a kind of cooperative setting. Is there a known way to set up transformer to do infix generation? That is: give it a start and end prompt, and an estimated number of tokens to fill in between. That seems like it should be doable and could improve things, but I haven't found any work on this problem yet and haven't had the time (and potentially don't have the skills) to look deeply myself yet.
- YeGoblynQueenne 7y ago
  Note that "just a few sentences" is more coherence than anything has ever managed before. Also, in many examples that I've seen there's a clear thread that runs through the generated text. For example, a couple small passages posted by others in this thread all revolve around one or two words that are repeated throughout the text, even if the details around those words keep changing See: https://news.ycombinator.com/item?id=21456705 https://news.ycombinator.com/item?id=21456705 Now it's "my bed" now it's a "strange bed" but it's always about a bed.
FillardMillmore 7y ago
As another mentioned, these models currently can't maintain a believably cohesive train of thought for any longer than 3 or 4 sentences. They are great at drawing statistical probabilities related to what words best conclude sentences and how punctuation should properly be used, but thus far have proven quite lacking in the ability to replicate true human creativity. The Economist ran an excellent article on this very concern recently: https://www.economist.com/books-and-arts/2019/10/31/dont-fear-the-writernator https://www.economist.com/books-and-arts/2019/10/31/dont-fea...
- derision 7y ago
  what if I used the agent to generate 1 paragraph at a time, and did my own editing after the fact to clean it up?
  - FillardMillmore 7y ago
    Might be possible. But I'd imagine the amount of time you'd save vs. just writing it yourself might be small to negligible.
minimaxir 7y ago
The next frontier is conditional generation. The CTRL model (https://github.com/salesforce/ctrl https://github.com/salesforce/ctrl) with a similar architecture to GPT-2 emphasizes conditional generation (e.g. generate a news article based on a URL) and the results are pretty good: https://minimaxir.com/2019/09/ctrl-fake-news/ https://minimaxir.com/2019/09/ctrl-fake-news/
taneq 7y ago
I'd say generating an entire coherent essay like this would constitute a solid pass of the Turing test. I don't think we're that close to that yet (or alternately the singularity is right around the corner, because you could feed that same bot a corpus of AI papers and have it write new publishable ones.)
- alain94040 7y ago
  But it should be possible to separately train a neural network on basic knowledge, so it can generate a stream of ideas. Then use GPT-2 to generate a sentence or paragraph, following those guidelines. Just like we humans: we first think of what we want to say (a few points about X, first Y, don't forget Z). Then turn those rough ideas into fully-formed sentences.
  - ebj73 7y ago
    Maybe it's worth a try, in order to advance science. But, generally, I think that text synthesis ultimately is one of those problems that we really cannot hope to solve before we've also solved the problem of creating full blown artificial general intelligence. When we have a system that actually understands both the world and the human beings in it, then we can probably also have a system capable of writing intelligent essays. Before that, we'd just be fiddling with different degrees of extremely stupid, I think.
ebj73 7y ago
I think it's very far off. You can see how this model drifts off into gibberish after only 3 or 4 words. It does not really understand the topic, even within a sentence, and much less within a whole paragraph. For it to understand the topic throughout a whole essay, that would probably require it to have full general intelligence on par with humans. And that's very, very far into the future, still.
TaupeRanger 7y ago
We are very far off, unless plagiarization is not considered "off the table".

Surely we are not far off models capable of submission-quality essays that will enable a new generation of cheating.

by chaz6 7y ago

Surely we are not far off models capable of submission-quality essays that will enable a new generation of cheating.

From my observation, even the largest GPT-2 model has difficulty retaining any long-range relationship information. In the "unicorn" writing example that was published originally, the model 'forgets' where the researchers are (climbing a mountain versus being beside a lake iirc) after just a few sentences. Because of this, it's hard to imagine models of this type being able to write long-form coherent papers. Now if we could somehow constrain the generated text to conform to a predefined graph structure that isn't forgotten so quickly...

jsinai 7y ago
Maybe the problem is that most of these models seem to rely on sequential information (even the transformer needs this for forward generation of text) to encode long range information. But I can’t remember the last time I relied on sequentially remembering the ordering of tokens in order to complete an essay or hell even reply to an email. Structurally we retain some kind of hierarchical information (topic, places, names, events) about text. Is there any active research looking into text generation models which do this? Maybe some kind of query that is made in a learned vector space and which is not temporarily dependent but rather “spatially” - as in these are the facts about the text being generated so far.
- flancian 7y ago
  I'm interested on this as well. I have been trying to fine-tune GPT-2 on genre fiction to work as a sort of "fiction replicator". Stylistically it actually seems to do quite reasonably, but it lacks narrative cohesion. This problem, as you point out, is corpus agnostic. I thought of trying to keep track of characters and key interactions outside of the model, but I haven't figured out how to make these two models interact reliably -- outside of just having the first component generate prompts for the second model in a kind of cooperative setting. Is there a known way to set up transformer to do infix generation? That is: give it a start and end prompt, and an estimated number of tokens to fill in between. That seems like it should be doable and could improve things, but I haven't found any work on this problem yet and haven't had the time (and potentially don't have the skills) to look deeply myself yet.
YeGoblynQueenne 7y ago
Note that "just a few sentences" is more coherence than anything has ever managed before. Also, in many examples that I've seen there's a clear thread that runs through the generated text. For example, a couple small passages posted by others in this thread all revolve around one or two words that are repeated throughout the text, even if the details around those words keep changing See: https://news.ycombinator.com/item?id=21456705 https://news.ycombinator.com/item?id=21456705 Now it's "my bed" now it's a "strange bed" but it's always about a bed.

FillardMillmore 7y ago

As another mentioned, these models currently can't maintain a believably cohesive train of thought for any longer than 3 or 4 sentences. They are great at drawing statistical probabilities related to what words best conclude sentences and how punctuation should properly be used, but thus far have proven quite lacking in the ability to replicate true human creativity. The Economist ran an excellent article on this very concern recently: https://www.economist.com/books-and-arts/2019/10/31/dont-fear-the-writernator https://www.economist.com/books-and-arts/2019/10/31/dont-fea...

derision 7y ago
what if I used the agent to generate 1 paragraph at a time, and did my own editing after the fact to clean it up?
- FillardMillmore 7y ago
  Might be possible. But I'd imagine the amount of time you'd save vs. just writing it yourself might be small to negligible.

minimaxir 7y ago

The next frontier is conditional generation. The CTRL model (https://github.com/salesforce/ctrl https://github.com/salesforce/ctrl) with a similar architecture to GPT-2 emphasizes conditional generation (e.g. generate a news article based on a URL) and the results are pretty good: https://minimaxir.com/2019/09/ctrl-fake-news/ https://minimaxir.com/2019/09/ctrl-fake-news/

taneq 7y ago

I'd say generating an entire coherent essay like this would constitute a solid pass of the Turing test. I don't think we're that close to that yet (or alternately the singularity is right around the corner, because you could feed that same bot a corpus of AI papers and have it write new publishable ones.)

alain94040 7y ago
But it should be possible to separately train a neural network on basic knowledge, so it can generate a stream of ideas. Then use GPT-2 to generate a sentence or paragraph, following those guidelines. Just like we humans: we first think of what we want to say (a few points about X, first Y, don't forget Z). Then turn those rough ideas into fully-formed sentences.
- ebj73 7y ago
  Maybe it's worth a try, in order to advance science. But, generally, I think that text synthesis ultimately is one of those problems that we really cannot hope to solve before we've also solved the problem of creating full blown artificial general intelligence. When we have a system that actually understands both the world and the human beings in it, then we can probably also have a system capable of writing intelligent essays. Before that, we'd just be fiddling with different degrees of extremely stupid, I think.

ebj73 7y ago

I think it's very far off. You can see how this model drifts off into gibberish after only 3 or 4 words. It does not really understand the topic, even within a sentence, and much less within a whole paragraph. For it to understand the topic throughout a whole essay, that would probably require it to have full general intelligence on par with humans. And that's very, very far into the future, still.

TaupeRanger 7y ago

We are very far off, unless plagiarization is not considered "off the table".