12 ms·
Human or Machine: Can You Tell Who Wrote These Poems?
- blowski 10y agoPoetry that scans correctly should be fairly easy to write for a machine - just find words with the right rhythm patterns and rhymes. Poetry that connects on an emotional level must be a lot harder.
- andreaferretti 10y agoI was tricked by "A thousand pictures on the kitchen floor, Talked about a hundred years or more."
- gridspy 10y agoI got 6/6 I didn't even read that far. It was obvious by here: SONNET #2 The dirty rusty wooden dresser drawer. A couple million people wearing drawers, Or looking through a lonely oven door, Flowers covered under marble floors. This is exactly how a computer algorithm would conflate drawer with drawers. There is no obvious theme to this verse, no high level construct. Every human poem in this example has a high level theme, which is non-obvious to a computer. None of the computer ones do.
- chrisfosterelli 10y agoThat's how I got 6/6 as well. It's less about the actual words and more about if the poem was actually talking about anything coherent.
- mfoy_ 10y agoAll the computer poems were just rambling and rhyming and weren't really even trying to say anything. A lot of the time they phrased things VERY awkwardly too, to the point of being incomprehensible.
- elorant 10y agoSounds like rap music :)
- plaguuuuuu 10y agoAlso interesting to compare with less thematically structured work. Probably not the greatest example but what comes to my mind is some of The Mars Volta's lyrics; a lot of it is pretty abstract and stream-of-consciousness and it's hard to really pinpoint a narrative as such, compared to the example poems which were pretty straightforward "Last night I heard lepers flinch like birth defects its musk was fecal in origin as the words dribbled off of its chin it said I'm lost I'm lost Now I'm lost Dolls wreck the minced meat of pupils cast in oblong arms length the hooks have been picking their scabs where wolves hide in the company of men it said I'm lost I'm lost"
- Scriptor 10y agoYep, the human poems would often refer back to an idea or word mentioned before. The computer poems read like something from a good markov generator. Actually, you can kind of reverse engineer the algorithm behind Sonnet #2. It seems like they first pick a bunch of related words that rhyme (drawer, door, floors, apartment, wall) and then build up the line backwards. The problem is that even though the words are related, the actual lines don't form any single cohesive image.
- SilasX 10y ago>This is exactly how a computer algorithm would conflate drawer with drawers. Because human poets never do wordplay like that?
- gridspy 10y agoBecause when humans do, there is an obvious (sometimes less obvious, but present) reason to do so. There is no such link made in this example between the usages.
- SilasX 10y agoYou haven't read a lot of schoolkid poetry posted at public libraries, have you?
- neosat 10y agoI got this one wrong. I thought no way a computer would be so dumb to have drawer and then drawers in the next sentence and this was a trick question to get one to say machine. Guess I overestimated the algorithm !
- taneq 10y agoInability to differentiate homonyms is one of the current weaknesses of word embeddings, isn't it?
- p4wnc6 10y agoBut like most art, whether it connects on an emotional level is mostly dependent on the consumer. Once an artist gains fame, then it becomes a matter of fashion signalling to agree with others about how good that artist is or to deride that artist for making something that doesn't live up to previous works. I found a lot of the lines from The Policeman's Beard is Half-Constructed by the program Racter to connect with me on an emotional level, despite knowing that (a) it was likely heavily edited by the program creator and (b) much of it just mimicked the unusual word choices and rhythmic qualities of poetry, not the intentional conveyance of a message or theme.
- gridspy 10y agoThat's the interesting thing about looking through the output of any computer algorithm. I have often done this with random walks using markov chains. By editing 200 lines into just 3 or 4, you are actually adding a lot of information. It makes the computer seem smarter than it is. If we could create a "good poem" evaluator + editor and tack it on the end of a "random poem" creator, we could perhaps get some good results.
- gridspy 10y agoThat's the interesting thing about looking through the output of any computer algorithm. I have often done this with random walks using markov chains. By editing 200 lines into just 3 or 4, you are actually adding a lot of information. It makes the computer seem smarter than it is. If we could create a "good poem" evaluator + editor and tack it on the end of a "random poem" creator, we could perhaps get some good results.
- TheOtherHobbes 10y ago>But like most art, whether it connects on an emotional level is mostly dependent on the consumer. Consumers. The best art affects a lot of people. And they don't need to be told to like it. It "just works." Art audiences follow multivariate statistical distributions. It's very much not just a personal reaction - it's a personal reaction shared by many other people having broadly similar reactions. Some of it is social signalling, and some of it requires education and experience. But valued works tend to persist in culture because they reliably create reactions in a statistically significant proportion of all possible audiences.
- SilasX 10y agoRight, because you have to convince the high-status authorities to say that it connects on an emotional level so that everyone else will nod their heads to avoid looking stupid. Truly, AI is a long way from beating the Emperor's New Clothes game. But frankly, there are easier, more useful AI goals to target.
- mfoy_ 10y agoI'm guessing you don't like poetry.
- justratsinacoat 10y ago>Right, because you have to convince the high-status authorities to say that it connects on an emotional level so that everyone else will nod their heads to avoid looking stupid Honest question -- did you get the same emotional read off of the machine-poems as you did the human-poems? If not, perhaps there is something more involved in poetry than stringing words together!
- SilasX 10y ago>Honest question -- did you get the same emotional read off of the machine-poems as you did the human-poems? Yep -- none.
- ythl 10y agoHmm, well I was able to correctly identify every single sonnet. The machine generated ones I would read and think "What the heck was that poem about? It just rambled on about random things" whereas the human ones you could tell that all the verses were interconnected. Take Sonnet #6 for example. You can tell there is intelligence behind the sentences being written. Each line builds on the line before it. Compare to sonnet #5 which rambles on about nothing and uses duplicate words (then comes a small bright spark comes)...
- SilasX 10y agoYou've never seen a human poem that just seemed to ramble about nothing?
- justratsinacoat 10y ago>Poetry that scans correctly should be fairly easy to write for a machine - just find words with the right rhythm patterns and rhymes Indeed, Pentametron [0] ('With algorithms subtle and discreet/I seek iambic writings to retweet') has been up since 2012. Despite the occasional serendipitous pairing, it obviously doesn't make much sense, as didn't the obvious machine-poems in TFA. [0]https://twitter.com/pentametron https://twitter.com/pentametron
- thaumasiotes 10y agoJust to point out, human poems, and in fact the poems here, often don't scan or rhyme correctly.
- thebladerunner 10y agoThis is another variation of the Turing test. Time to realize: the Turing test is useless because: (a)the goal is to game the jury and (b) humans don't necessarily make sense either (example: bad or even just confusing poetry) so the human baseline is not always well-defined. Need Turing Test 2.0
- thaumasiotes 10y agoI found a website once advertising "have a conversation with me or my chatbot, and try to guess who it is". My counterparty quickly made some bizarre grammatical errors, prompting me to guess "chatbot" and be rewarded with a screen saying "No, you fool! See, telling the difference is harder than you think!" Telling the difference between a robot and a human pretending to be that robot is apparently viewed as a Turing test, but it isn't actually the same thing. Similarly, there's a Turing test that's run every year with very strict rules on what you can and can't say to the other side. This defeats the idea of the test (while making the event less nonsensical, in that the rules allow for people to misidentify chatbots some of the time). I guess my point is, your point (a) is an artifact of a certain culture that's trying to appropriate the concept of a Turing test so that they can declare they've beaten it. Point (b) is better.
- p1esk 10y agoTuring Test is a personal experience. For example, I will only believe TT has been passed if I personally had a chance to talk to the "Entity Under Test", and I can't tell it's human or not. I don't really care whether other people have been fooled or not. Obviously, the EUT should be, or pretend to be, my intellectual peer (e.g. similar education, age, social status, country, etc). It should be a native English speaker, or at least as fluent as I am. I should be able to converse with EUT for as long as I need to (several conversations if needed, perhaps limited to a couple of weeks). Most importantly, the human participants should be motivated to convince me they are human (e.g. they only get paid if they manage to do so).
- taneq 10y agoA Turing test is meant to be both the AI and the human trying to convince the tester that they're human. To do it properly your human testee should be a third party not related to the AI developer, and should be independently motivated to 'win' by convincing the tester that they're the human. Having the human pretend to be a chatbot is just disingenuous.
- thebladerunner 10y agoThis is another variation of the Turing test. Time to realize: the Turing test is useless because: (a)the goal is to game the jury and (b) humans don't necessarily make sense either (example: bad or even just confusing poetry) so the human baseline is not always well-defined. Need Turing Test 2.0
- daveguy 10y agoAs others have mentioned, lack of an overall theme is an easy way to distinguish the human from machine poetry. Keep in mind, the machine poems are those hand selected by humans, twice filtered -- this publication and the original publication. That means these poems have a high degree of coherency that you wouldn't otherwise see. People are reporting 6/6 ability to distinguish. If you had a person and an algorithm each generate two (or even one) poem without any filtration from a field of poems then it would be even more obvious which was machine and which was human.
- TheOtherHobbes 10y agoUnfortunately the problem reduces to qualia. Poems that have a theme evoke the experience of that theme. Likewise for generative music and art. If you have no model for experience (it doesn't have to be metaphysical - something mechanical would do) you can't include that element in the output. Except by accident - which is how Deep Dream works. The designers found a model that was good at mimicking a certain kind of experience - partly through luck, and partly through analogy. But Deep Dream is limited to that one kind of trippy experience. It's very good at it, but a competent human artist has a much wider range. [1] Real AI creativity needs a system that can mimic that much wider range - so you'll never be sure what kind of human experience you'll get from it. [1] Sometimes. A surprising proportion of professional artists repeat the same theme, style, or imagery over and over. But that's sometimes because of commercial pressure - when you find a formula that sells, experimenting with a formula that might not isn't always wise.
- sandworm101 10y agoPoetry is the literary equivalent of minimalist painting. The machine is doing just enough, or as little as possible to tick the boxes needed to make the art identifiable as art. But art only exists in context. These poems' only value is that they have come from a machine, just as many paintings derive most of their value from the person who paints them. They are interesting, but as art they are next to worthless because the process of their creation is so easily replicated. This is art: https://en.wikipedia.org/wiki/File:IKB_191.jpg https://en.wikipedia.org/wiki/File:IKB_191.jpg
- sid- 10y agoMaybe we should have a machine learning algorithm do this ? :)
- suprgeek 10y agoI see a future where as the "rise of the machines" takes hold in things like News Items, Stories, Poetry, Paintings, Movies etc, there will be a commensurate rise in Detection software. The detection software would flag things as human-generated vs machine-generated. Pretty soon anything created by Humans will be intrinsically more valuable just because it was human-created and hence somehow better than "machine-generated". We live in interesting times...
- beloch 10y ago6/6, and I didn't read more than half-way through most of these. What's the tell? Focus. The human-written poetry develops a single, coherent subject with each passing line. Each new line adds new light to the ones that preceded it. Any single line of the machine written sonnets, considered alone, might appear to be written by a human but, when taken together, they simply don't add up. The progression of a single idea just isn't there. I freakin' hate poetry. Give me prose or give me epigrams! To me, a poem is an awkward combination of prose without clarity and and epigram too bloated to be stated succinctly. I point this out, because I want to be taken seriously when I say that this machine-written poetry is bad in a way that makes me appreciate human poetry just a little bit more.
- kazinator 10y agoSame here! 6/6 within five minutes. The computer ones are just laughable gibberish, reminiscent of word salad generation from some 1993 dated Perl tutorial.
- deleted 10y ago[deleted]
- maze-le 10y ago> The human-written poetry develops a single, coherent subject with each passing line Yep, thats it. The machine-written poems make seemingly correct sentences, but fail to create a coherent sense. Its quite easy to spot. The machine-written parts use: The Pythonic Poet (http://ischoolreview.com/iSR_Grav/entries/entry-6 http://ischoolreview.com/iSR_Grav/entries/entry-6). It works with markov-chains and support-vector machines.
- madmax96 10y agoSame, it took me just a few minutes. The human writers all have a point, whereas the machine writers (although they sometimes do have some beautiful imagery) obviously lack a plot or overall structure. It's just words without purpose.
- adrusi 10y ago
- kamingme 10y agoBut if there aren't human examples that machine couldn't write poems. They are just mimicking within the samples and structures.
- cel1ne 10y agoI read only the ending rhyme of every sonnet => 6/6.
- suprfnk 10y agoI really doubt if you only read this sentence (last sentence of sonnet #2) you'd know it's a machine: A thousand pictures on the kitchen floor, Talked about a hundred years or more. If you don't know what's written in the preceding sections, this could very well be written by a human.
- cel1ne 10y agoOh you're right, I read upwards at that poem and looked at the ending words of the sentences. Decided that the combination "tripping" / "dripping" wasn't very poetic. -> machine.
- khrm 10y agoI find poems given at http://botpoet.com http://botpoet.com by bots to be a bit better. Above one are completely incoherent. Haiku would be probably more easier to generate.
- aaron695 10y agoIf a machine writes 1000 poems and a human chooses the best one did the machine make a poem with emotion or a human?