4 ms·

.....MAN, I am not sure how I feel about this. Don't take this the wrong way, it definitely looks like you are just using technology to produce art/craft in you

• nullc 6y ago
  I just tried GPT2 at cover letters and I can't figure out a prompt that gets it to consistently write good ones, instead I can mostly get it to write bad ones (as in the sort of stuff you might see unsuccessful applicants submit). My example problem was to get it to apply to Y Combinator. It managed to misspell the organization which isn't something GPT2 normally does. Here are some examples: https://0bin.net/paste/RzN1n+cubZwvHDKc#hV5+TXf4eNRRB9ROuXIT7ILb22et7985WPc+A9zD6t1 https://0bin.net/paste/RzN1n+cubZwvHDKc#hV5+TXf4eNRRB9ROuXIT... After the first time it produced "Dear Paul Graham," I stuck it on the end of my prompt as it tended to reliably produce a cover letter (and one for YC) rather than a blank job application form. Other than throwing lots of computing power at it: I find that it's useful to include some largely irrelevant flavour text in your prefixes that you can stir until you hit on a combination that triggers the results you want (seemingly random things will send the model off in vastly different directions). When it has good initial output but goes off the rails to you can just terminate it at the last good part and continue from there-- a lot of GPT2's worst sins can be correct with fairly low effort that way. As mentioned, I think these examples are mostly pretty poor. I suspect that if I worked on it I could probably hit on a prompt that caused it to generate better cover letters... That said, there are perhaps a few things I might not have considered without reading them.
  • saberience 6y ago
    Maybe this is a dumb question, because you seem like someone who thinks in a radically different way to me. But why would you try and use GPT-2 to help you write better English versus actually learn to become a better writer yourself? I say this as someone who reads extensively and is considered a "good" writer and have won prizes before for my writing. I became a better writer through reading more books (especially classic literature) and writing more. Cribbing off a computer algorithm and copying and pasting seems like it would just be hurting your own growth, as well as being lazy to boot.
    • nullc 6y ago
      I don't care that much about being a 'good writer' at least not according to any abstract sense of good. I care more about exploring interesting ideas and interesting interactions, joyful leaps of insight, surprise and serendipity. Like in any other art just being different can have its own merit. The nearest I usually come to caring about good is that at times I aspire to be an /effective/ writer. But in many cases I've found being an effective writer can requires making the far-out leap that no one else was making or it can depend on making good guesses at the minds of readers far different from my own. While lucid language and fully formed ideas are necessary tools, the spice of something unfamiliarly or too familiar-- an allusion, even a silly or vulgar one, that people can't unstick from their minds, or anticipating their thoughts so that your writing speaks in harmony with their inner voice-- can really help a message stand out. Some of the most effective writing, as I see it, comes from taking a step back from the words on the page and asking yourself "what does this actually say about the world?" Any tool that helps you adopt a different perspective can be useful to these ends. Working with the machine is not just copy-and-paste-- if it were, there would already be no more need for writers. It is its own art and one that has yet to be mastered by anyone. In particular tools like GPT2 are absurdly sensitive to the prompts. There is also skill and creativity that can go into sampling, knowing when tell it to take greater or lesser risk, where to cut it off and retry or which word to insert to move it back in a fruitful direction. Fine details like what you name objects, people, and places implicitly signal to the model the genre of the writing and help guide it down useful paths or into blind allies. We sometimes think about the connotations of the words we use but for the model the connotations are all that matter. People talk some about bias in AI but in some sense the model is nothing but biases. Strange biases, some alien and impervious to analysis, others are biting social commentary once you notice them. Writers have always used devices and tools of various forms-- narrative forms, patterns of speech, constrained writing, and so on-- and machine text generation is but one more. I suppose that there are more dull ways of using it than brilliant ones, but that is true of any tool. We might write with a pen or a typewriter, but the tool doesn't become the author simply because we used it to write. The same is true using the computer even if, at times, the line may be fuzzier. We also shouldn't kid ourselves: Good writing, however we define it, is both derivative and also dependant on a health dose of chance and luck. If all the machine did was give us another way to be derivative or another way of finding fortunate statements by chance we could still find value in that alone. I don't think it's likely that I'll be the first to discover the best ways to use machine-co-authorship to improve writing, if such ways are ever to be discovered. Neither is it likely that I'll become a renowned writer by traditional means. But the former currently has less competition and the latter has already been done. And besides, I find it fun.