4 ms·

I’m an Edgar Finalist, Wall Street Journal best-selling author, blah, blah, and I can confidently say that...I’m doomed. GPT-2 is amazing without even grasping

by amayne 7y ago

I’m an Edgar Finalist, Wall Street Journal best-selling author, blah, blah, and I can confidently say that...I’m doomed. GPT-2 is amazing without even grasping deeper context. Once NN’s start to grasp arc and continuity (which is understanding changes over time) I’ll be looking for a new job.

irishcoffee 7y ago
I realize I’d be giving you away, can I ask which book(s)?
- amayne 7y ago
  My username gives me away. My most recent series is The Naturalist. It's about a computational biologist that hunts serial killers. Even though the book goes into Bayesian statistics, Fourier analysis and has the occasional bit of Python, the series has been well received by the general public (it's Wall Street journal best seller and was optioned by a major studio.) Diving into ML for research for the book lead me into actually doing my own projects and current AI startup.
  - thewhitetulip 7y ago
    I've read that book! I specifically logged in to say this. I liked the book! As someone who wrote 6 unpublished novels, I'm envious of you that you got to publish that book (as in a story with a science touch)! I specifically remember the scene where the protagonist is looking for bodies on the hill and he identifies the dead bodies acc to the plants that grew. It positively surprised me (that some ppl ala detectives actually are capable of that level of thinking) and gave me the creeps every time I go to the forest :p. My books for rejected from every publisher and agent I sent it to and it is a duology about a "modern" dictatorship. Part 1 is a mystery genre Part 2 is a satire. Sadly, I live in India and as per an Editor "Love and historical fiction sells in India and yours ain't that" I've now given up.
    - amayne 7y ago
      Nobody knows anything. Harry Potter got rejected by over 40 publishers. I started as a self-published author and sold hundreds of thousands of books plus made my first movie deal before ever getting a publisher. I was actually better early-off as an indie than with some of my first publishing deals. Although I learned a lot by working with a first-rate editor. You don't need a publisher in 2019. You need a good book, a good editor, a good cover and a good description...and to keep writing. A year after I sat down to write my first self-published novel I was sitting across the table from one of my favorite directors on the Disney lot talking about my book. All of that because I started by self-publishing.
      - thewhitetulip 7y ago
        True that. Thank you for your input
  - areoform 7y ago
    Inside baseball question, does it help you more if I buy the book, or if I use Kindle Unlimited to read it? What's the more beneficial business arrangement for you?
    - amayne 7y ago
      Whatever is convenient for you. I'm just happy to have readers. The most beneficial thing it is to leave a review if you like it and tell people what you enjoyed.
lordgrenville 7y ago
Can't disagree with this enough. GPT-2 is good at generating coherent original sentences based on a given corpus, but they don't really mean anything, which rapidly becomes clearer with each extra sentence. Ask it for an essay and you get a string of sentences with zero underlying point. The distance from here to writing original novels is very, very far. (Experimental poets, however, might need to worry!)
- amayne 7y ago
  > Can't disagree with this enough. I'm not sure if you're actually disagreeing with me. My point is once some future neural network starts to grasp continuity and context (something GPT-2 does not do) I think writing won't be safe from AI. I couldn't speculate when that would be, but I've seen some unpublished efforts that make me think it's sooner than many people might think. But between here and there will be the opportunity for AI-assistive writing that could facilitate what my developmental editors and beta-readers do.
  - Hoasi 7y ago
    > My point is once some future neural network starts to grasp continuity and context (something GPT-2 does not do) I think writing won't be safe from AI. If this is so, writers will more likely become editors. In the same way, visuals artists might become curators. As an optimist, I am still pondering how this could be a good thing.
  - notahacker 7y ago
    I don't think it's at all guaranteed that future neural networks will grab continuity and context [in a meaningful sense, as opposed to basics like the character and place names being consistent and the context continuing to involve a murder and detectives] still less actually plot well. More importantly, for actual publishable writing as opposed to spam content AI has to do more than just start to grasp continuity and context, because every unpublishable human writer reaches that level, and most of them understand the much higher level of narrative arcs, characterisation and twists too. I don't think the cost of cash advances for no-name writers or lack of half decent unsolicited submissions to choose from is a big problem for publishers, is it? Surely 'time to fix not-irredeemably-flawed texts up enough to make them marketable' is in shorter supply... Agreed that AI-assistive writing is pretty close to the point where it can save you the hassle of thinking up interesting ways of describing minor characters and locations, but of course when it reaches that level there are massive areas of content creation for AI companies which are more lucrative than novel writing and don't require further technological advances...

by amayne 7y ago

irishcoffee 7y ago

I realize I’d be giving you away, can I ask which book(s)?

amayne 7y ago
My username gives me away. My most recent series is The Naturalist. It's about a computational biologist that hunts serial killers. Even though the book goes into Bayesian statistics, Fourier analysis and has the occasional bit of Python, the series has been well received by the general public (it's Wall Street journal best seller and was optioned by a major studio.) Diving into ML for research for the book lead me into actually doing my own projects and current AI startup.
- thewhitetulip 7y ago
  I've read that book! I specifically logged in to say this. I liked the book! As someone who wrote 6 unpublished novels, I'm envious of you that you got to publish that book (as in a story with a science touch)! I specifically remember the scene where the protagonist is looking for bodies on the hill and he identifies the dead bodies acc to the plants that grew. It positively surprised me (that some ppl ala detectives actually are capable of that level of thinking) and gave me the creeps every time I go to the forest :p. My books for rejected from every publisher and agent I sent it to and it is a duology about a "modern" dictatorship. Part 1 is a mystery genre Part 2 is a satire. Sadly, I live in India and as per an Editor "Love and historical fiction sells in India and yours ain't that" I've now given up.
  - amayne 7y ago
    Nobody knows anything. Harry Potter got rejected by over 40 publishers. I started as a self-published author and sold hundreds of thousands of books plus made my first movie deal before ever getting a publisher. I was actually better early-off as an indie than with some of my first publishing deals. Although I learned a lot by working with a first-rate editor. You don't need a publisher in 2019. You need a good book, a good editor, a good cover and a good description...and to keep writing. A year after I sat down to write my first self-published novel I was sitting across the table from one of my favorite directors on the Disney lot talking about my book. All of that because I started by self-publishing.
    - thewhitetulip 7y ago
      True that. Thank you for your input
- areoform 7y ago
  Inside baseball question, does it help you more if I buy the book, or if I use Kindle Unlimited to read it? What's the more beneficial business arrangement for you?
  - amayne 7y ago
    Whatever is convenient for you. I'm just happy to have readers. The most beneficial thing it is to leave a review if you like it and tell people what you enjoyed.

lordgrenville 7y ago

Can't disagree with this enough. GPT-2 is good at generating coherent original sentences based on a given corpus, but they don't really mean anything, which rapidly becomes clearer with each extra sentence. Ask it for an essay and you get a string of sentences with zero underlying point. The distance from here to writing original novels is very, very far. (Experimental poets, however, might need to worry!)

amayne 7y ago
> Can't disagree with this enough. I'm not sure if you're actually disagreeing with me. My point is once some future neural network starts to grasp continuity and context (something GPT-2 does not do) I think writing won't be safe from AI. I couldn't speculate when that would be, but I've seen some unpublished efforts that make me think it's sooner than many people might think. But between here and there will be the opportunity for AI-assistive writing that could facilitate what my developmental editors and beta-readers do.
- Hoasi 7y ago
  > My point is once some future neural network starts to grasp continuity and context (something GPT-2 does not do) I think writing won't be safe from AI. If this is so, writers will more likely become editors. In the same way, visuals artists might become curators. As an optimist, I am still pondering how this could be a good thing.
- notahacker 7y ago
  I don't think it's at all guaranteed that future neural networks will grab continuity and context [in a meaningful sense, as opposed to basics like the character and place names being consistent and the context continuing to involve a murder and detectives] still less actually plot well. More importantly, for actual publishable writing as opposed to spam content AI has to do more than just start to grasp continuity and context, because every unpublishable human writer reaches that level, and most of them understand the much higher level of narrative arcs, characterisation and twists too. I don't think the cost of cash advances for no-name writers or lack of half decent unsolicited submissions to choose from is a big problem for publishers, is it? Surely 'time to fix not-irredeemably-flawed texts up enough to make them marketable' is in shorter supply... Agreed that AI-assistive writing is pretty close to the point where it can save you the hassle of thinking up interesting ways of describing minor characters and locations, but of course when it reaches that level there are massive areas of content creation for AI companies which are more lucrative than novel writing and don't require further technological advances...