4 ms·

Knowing it's an AI that wrote it, it seems impressive. (Although I'm not sure how much human guidance it has). The premise is I think actually pretty good -- b

Knowing it's an AI that wrote it, it seems impressive. (Although I'm not sure how much human guidance it has). The premise is I think actually pretty good -- but maybe the premise and basic plot really comes from the human-written parts? The actual writing? If I didn't know an AI had created it and thought a human had, I'd say the writing is pretty terrible. At the sentance level and at the story level, it's just pretty poorly written and hard to follow, with weird choices. But... sure, maybe it's too much to expect an AI to write a well-written story at this point, the fact that it seems to be a somewhat coherent story at all is the point? OK if so, but yeah.

andreyk 4y ago
wrt "(Although I'm not sure how much human guidance it has)" - at the top it says "Text Formatting: Human-written text is italic, AI-generated text is normal" so that kind of demonstrates the ratio [and the human selects the AI completions among a few options each time] And yeah, generally agree - as we say on our About page, these stories are sort of explorations/demonstrations of what AI can do, as much as they are attempts at entertaining writing. So in a sense we want to keep the "weird choices" AI makes to let the story go in weird directions. Since we release weekly, the intent is in general not to go for a lot of polish but rather for a fun concept and just an entertaining read.
- jrochkind1 4y ago
  Fair. But i'm not talking as much about the story "going in weird directions", as I am about the way the sentences and paragraphs are constructed. Or apparent contradictions like "We found that planets with life on them were rare" followed by "But life did develop on one planet" -- so is that "rare" or just one? "which evolved one of the most diverse biospheres we had ever seen" -- wait, so how many "biospheres" (implying life) have you seen, again? "one of the most... we've seen" is a weird thing to say if you've seen only one, or even only a few! It's just like, wait, what are you trying to tell me here, what parts of these sentances serve the story? Is the feedback I'd give if this were a writing workshop, I'm not talking about the way the plot developed, I'm talking about the way the words are constructed into sentances and sentances into paragraphs. It just seems... sloppy, like someone wasn't trying very hard just kind of throwing words on a page without thinking very much and not editing. But it is all understandable more-or-less grammatical English which is a coherent (if poorly-written) narrative, so, sure, that's a thing.
  - andreyk 4y ago
    To your last paragraph, yeah that's kind of the case. We want to use the AI generated as-is more or less to showcase what it produces, so editing those sentences you mention to be more consistent with each other would mess with that. Plus, trying to finish a short story once per week while working a full time job (we currently have one main author on the team) kind of necessitates rushing things and not polishing that much. So the intent is not to write a particularly well-written narrative so much as a fun/entertaining read that showcases what the AI can bring to writing today (even if it's flawed, as you say). Perhaps we'll aim for more polished writing or more human editing in the future, we are still figuring things out as we go!
    - jrochkind1 4y ago
      I am not critisizing you, I am giving my own review of what the AI you are using has or hasn't accomplished in this case! I agree that you manually editing to be better written wouldn't have much purpose. I think the point of this exersize is probably "See what an AI can accomplish?" I was reviewing that.
      - andreyk 4y ago
        ah I see, I did not get that it seems. Cool, yeah I think your analysis of the AI's output is quite solid.
    - GalahiSimtam 4y ago
      Well this reminds me of the priming that is in AI Test Kitchen https://aitestkitchen.withgoogle.com/how-lamda-works https://aitestkitchen.withgoogle.com/how-lamda-works I mean, the example with "I'm at a planet made of marshmallows" and how the generated text changes from the default I guess style to a bombastic description. I don't have an example of that in GPT-3 playground right away, but I suppose it works similarly and could be used to nudge the AI writing style in some way. This is also why I'm not holding any expectations with offering you a story idea; the AI can come up with a story idea, in my experience, once I did not start with "You wake up in a cart in Skyrim" but with mere "You wake up in a carriage" and the AI started to improv right away, "A man in black robe enters the carriage", and I let it continue, chiefly on its own. Though I haven't saved that story, I will remember it well - the mysterious man in black robe locked up the protagonist with a book containing every knowledge of the universe, telling that after a year he'll come back to ask questions; and he came back after a year and asked Deep Philosophical Questions (TM), and the protagonist had not to shabby answers to them; among them stoically accepted death. Subsequently, the mysterious man took the book and left, leaving the room no longer locked. The protagonist traveled to the nearest village and settled to a work as a teacher. Fast forward twenty years later, meeting the black robed figure again, this time it was revealed to be Azrael (the angel of death); only then the protagonist felt truly the fear of death and that changed at least one of the answers to the Deep Philosophical Questions (TM) from the earlier part of the story. Feel free to use that as a prompt; I doubt the AI will recreate that story with a good writing style with just that, but AI are capricious and one never knows until tries.
joe_the_user 4y ago
This kind of summarizes my impression. I'd add that the story has a style I've come to expect from GPT-3. Basically, sequence of superlative statement alternating with fairly vacuous "softeners" - "I was in the lab the day that the greatest discovery in the history of science was made (though I acknowledge that I may be biased)." ... "But life did develop on one planet, which evolved one of the most diverse biospheres we had ever seen..." If it was a human writer, I'd say that the breathless tone makes it exhausting to read. And the lack of simple, "fleshing out" sentences means that, for example, it talks a lot of being excited by testing "their theories" but doesn't ever say that those theories are. And lots of human-written stories also do that. Many science fiction writers don't know modern physics in any detail and don't work out their science-variations in great detail. But naturally this system doesn't know anything but associations between words and makes up for this lack by using those associations quite effectively.
- visarga 4y ago
  > it talks a lot of being excited by testing "their theories" but doesn't ever say that those theories are. And lots of human-written stories also do that Good news, in a few years you can open your favourite book in GPT-3's successor and ask it to flesh out those theories. When AI will be able to write stories, all our pet peeves can be tried out, in any context.
- notahacker 4y ago
  The first sentence you quoted was part of the prompt (But yes, I thought the human wrote like an AI too. If nothing else, it achieved stylistic consistency)
visarga 4y ago
> maybe it's too much to expect an AI to write a well-written story at this point You just gave a great example of feedback. This kind of data can be used to train the model to generate self evaluation, maybe a few thousand samples would suffice to fine tune a large model. Then we could condition on good feedback.
- jrochkind1 4y ago
  I am not sure what you mean. You are talking about literally taking my comment as written verbatim and somehow feeding it back into the model? Can you say more about how this works?

andreyk 4y ago

wrt "(Although I'm not sure how much human guidance it has)" - at the top it says "Text Formatting: Human-written text is italic, AI-generated text is normal" so that kind of demonstrates the ratio [and the human selects the AI completions among a few options each time] And yeah, generally agree - as we say on our About page, these stories are sort of explorations/demonstrations of what AI can do, as much as they are attempts at entertaining writing. So in a sense we want to keep the "weird choices" AI makes to let the story go in weird directions. Since we release weekly, the intent is in general not to go for a lot of polish but rather for a fun concept and just an entertaining read.

jrochkind1 4y ago
Fair. But i'm not talking as much about the story "going in weird directions", as I am about the way the sentences and paragraphs are constructed. Or apparent contradictions like "We found that planets with life on them were rare" followed by "But life did develop on one planet" -- so is that "rare" or just one? "which evolved one of the most diverse biospheres we had ever seen" -- wait, so how many "biospheres" (implying life) have you seen, again? "one of the most... we've seen" is a weird thing to say if you've seen only one, or even only a few! It's just like, wait, what are you trying to tell me here, what parts of these sentances serve the story? Is the feedback I'd give if this were a writing workshop, I'm not talking about the way the plot developed, I'm talking about the way the words are constructed into sentances and sentances into paragraphs. It just seems... sloppy, like someone wasn't trying very hard just kind of throwing words on a page without thinking very much and not editing. But it is all understandable more-or-less grammatical English which is a coherent (if poorly-written) narrative, so, sure, that's a thing.
- andreyk 4y ago
  To your last paragraph, yeah that's kind of the case. We want to use the AI generated as-is more or less to showcase what it produces, so editing those sentences you mention to be more consistent with each other would mess with that. Plus, trying to finish a short story once per week while working a full time job (we currently have one main author on the team) kind of necessitates rushing things and not polishing that much. So the intent is not to write a particularly well-written narrative so much as a fun/entertaining read that showcases what the AI can bring to writing today (even if it's flawed, as you say). Perhaps we'll aim for more polished writing or more human editing in the future, we are still figuring things out as we go!
  - jrochkind1 4y ago
    I am not critisizing you, I am giving my own review of what the AI you are using has or hasn't accomplished in this case! I agree that you manually editing to be better written wouldn't have much purpose. I think the point of this exersize is probably "See what an AI can accomplish?" I was reviewing that.
    - andreyk 4y ago
      ah I see, I did not get that it seems. Cool, yeah I think your analysis of the AI's output is quite solid.
  - GalahiSimtam 4y ago
    Well this reminds me of the priming that is in AI Test Kitchen https://aitestkitchen.withgoogle.com/how-lamda-works https://aitestkitchen.withgoogle.com/how-lamda-works I mean, the example with "I'm at a planet made of marshmallows" and how the generated text changes from the default I guess style to a bombastic description. I don't have an example of that in GPT-3 playground right away, but I suppose it works similarly and could be used to nudge the AI writing style in some way. This is also why I'm not holding any expectations with offering you a story idea; the AI can come up with a story idea, in my experience, once I did not start with "You wake up in a cart in Skyrim" but with mere "You wake up in a carriage" and the AI started to improv right away, "A man in black robe enters the carriage", and I let it continue, chiefly on its own. Though I haven't saved that story, I will remember it well - the mysterious man in black robe locked up the protagonist with a book containing every knowledge of the universe, telling that after a year he'll come back to ask questions; and he came back after a year and asked Deep Philosophical Questions (TM), and the protagonist had not to shabby answers to them; among them stoically accepted death. Subsequently, the mysterious man took the book and left, leaving the room no longer locked. The protagonist traveled to the nearest village and settled to a work as a teacher. Fast forward twenty years later, meeting the black robed figure again, this time it was revealed to be Azrael (the angel of death); only then the protagonist felt truly the fear of death and that changed at least one of the answers to the Deep Philosophical Questions (TM) from the earlier part of the story. Feel free to use that as a prompt; I doubt the AI will recreate that story with a good writing style with just that, but AI are capricious and one never knows until tries.

joe_the_user 4y ago

This kind of summarizes my impression. I'd add that the story has a style I've come to expect from GPT-3. Basically, sequence of superlative statement alternating with fairly vacuous "softeners" - "I was in the lab the day that the greatest discovery in the history of science was made (though I acknowledge that I may be biased)." ... "But life did develop on one planet, which evolved one of the most diverse biospheres we had ever seen..." If it was a human writer, I'd say that the breathless tone makes it exhausting to read. And the lack of simple, "fleshing out" sentences means that, for example, it talks a lot of being excited by testing "their theories" but doesn't ever say that those theories are. And lots of human-written stories also do that. Many science fiction writers don't know modern physics in any detail and don't work out their science-variations in great detail. But naturally this system doesn't know anything but associations between words and makes up for this lack by using those associations quite effectively.

visarga 4y ago
> it talks a lot of being excited by testing "their theories" but doesn't ever say that those theories are. And lots of human-written stories also do that Good news, in a few years you can open your favourite book in GPT-3's successor and ask it to flesh out those theories. When AI will be able to write stories, all our pet peeves can be tried out, in any context.
notahacker 4y ago
The first sentence you quoted was part of the prompt (But yes, I thought the human wrote like an AI too. If nothing else, it achieved stylistic consistency)

visarga 4y ago

> maybe it's too much to expect an AI to write a well-written story at this point You just gave a great example of feedback. This kind of data can be used to train the model to generate self evaluation, maybe a few thousand samples would suffice to fine tune a large model. Then we could condition on good feedback.

jrochkind1 4y ago
I am not sure what you mean. You are talking about literally taking my comment as written verbatim and somehow feeding it back into the model? Can you say more about how this works?