11 ms·
The 'creepy Facebook AI' story that captivated the media
- deleted 9y ago[deleted]
- spyspy 9y agoMakes sense. Bot-0 creates a message that's slightly incorrect to a human reader. Bot-1 reads and interprets it correctly. Bot-0 gets a reasonable response from Bot-1 and thus increasing the confidence in how it structured its original message. Millions of iterations later and you get a weird corpus with weird structure. You can achieve the same thing with two young children that spend a lot of time together e.g. twins.
- digi_owl 9y agoAnd if you scale it up, dialects and even languages.
- mrec 9y ago> You can achieve the same thing with two young children that spend a lot of time together e.g. twins. Sometimes taken to extremes: https://en.wikipedia.org/wiki/Cryptophasia https://en.wikipedia.org/wiki/Cryptophasia
- chillee 9y agoWow the linked article is quite something: https://en.wikipedia.org/wiki/June_and_Jennifer_Gibbons https://en.wikipedia.org/wiki/June_and_Jennifer_Gibbons > According to Wallace, the girls had a longstanding agreement that if one died, the other must begin to speak and live a normal life. During their stay in the hospital, they began to believe that it was necessary for one of them to die, and after much discussion, Jennifer agreed to be the sacrifice.[4] In March 1993, the twins were transferred from Broadmoor to the more open Caswell Clinic in Bridgend, Wales; on arrival Jennifer could not be roused.[5] She was taken to the hospital where she died soon after of acute myocarditis, a sudden inflammation of the heart.[5] There was no evidence of drugs or poison in her system, and her death remains a mystery.[6] At the inquest, June revealed that Jennifer had been acting strangely for about a day before their release, her speech was slurring, and she said that she was dying. On the trip to Caswell, she had slept in June's lap with her eyes open.[3][7] On a visit a few days later, Wallace recounted that June "was in a strange mood". She said, "I'm free at last, liberated, and at last Jennifer has given up her life for me".[5]
- digi_owl 9y agoSo one of them actually willed herself to die?!
- SmellyGeekBoy 9y agoOr she had been experiencing chest pain and had a pretty good idea that she was going to die anyway.
- oliv__ 9y agoMy question (might be dumb, I don't know much about ML) here is: how do you know that the bots are actually understanding each other as opposed to just sending each other utter gibberish and responding like everything is fine? In other words, how do you determine that there is meaning in a conversation spoken in a language you don't understand?
- cosinetau 9y agoDoes anyone else think that journals, and periodicals misrepresenting something like this is getting to be too much like journalistic terrorism? Or would calling them Luddites be a better characterization?
- tpeo 9y agoI wouldn't go as far as calling it either terrorism or Luddism. Because, frankly, it doesn't seem like they care. It's just sensationalism.
- arcanus 9y agoI've worked with non-technical writers for some of my research. The problem is not that these people are not smart, but they tend to be very superficial. Reporting seldom rewards (or attracts) individuals who are interested in deep dives of anything. The culture is that of hot takes and fast ideas, which necessarily has resulted in a race to the bottom of content and meaning. That is where this stuff comes from. It might be 'journalistic terror', but barring a total reboot of the field, I don't see how it will change.
- wonderwonder 9y agoI don't think it's either. I think that for many now journalism is reduced to counting clicks and selling ads. This is why legitimately "fake news" has gained so much sway especially on social networks. Print a story with a headline that people want to read and they will click on it, truth takes a backseat to currency. Easiest way to do that is to write something supporting someones deeply held belief or fear. Now I don't think that the major news organizations purposely publish fake stories but as another poster said, a little sensationalism while conveying an otherwise truthful story helps to encourage clicks and improves a papers bottom line. With the ongoing demise of print and the rise of competition on the web, legitimate news organizations are attempting to compete and survive and often that means employing some of the methods that we find distasteful but are proven to work.
- lukas099 9y agoI'll bet an AI clickbait-generator could create a ton of revenue. I wonder what kind of sensationalist clickbait that would inspire?
- cwyers 9y agoThis is better than most articles I've seen on this subject, but it still falls victim to AI hype. > Although some reports insinuate that the bots had at this point invented a new language in order to elude their human masters, a better explanation is that the neural networks had simply modified human language for the purposes of more efficient interaction. From the Gizmodo article it links to (which is also okay, but not great): > “Agents will drift off understandable language and invent codewords for themselves,” FAIR visiting researcher Dhruv Batra said. “Like if I say ‘the’ five times, you interpret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.” Except... this isn't shorthand. "I want five balls" is shorter than "Give me balls the the the the the," or whatever else it could come up with. It's not more efficient. It's just... dumb. The bots don't actually understand the words the way humans do. Because they don't understand words and language, they're using the words as tokens. It's this primitive level of communication that just happens to have words assigned to it. Even the calmer takes on this are anthromorphizing the machines too much and attributing intelligence to the complete lack thereof.
- simias 9y agoIt depends what you're optimizing for I guess. If you want to reduce the length of the sequence then "Give me balls the the the the the" is silly but if the point is to reduce the vocabulary then it makes sense. After all most machines currently communicate by exchanging sequences of exactly two symbols, 0 and 1, not some complex phonology. Maybe when bandwidth is not as constrained as human speech it's actually more efficient to reduce the vocabulary and increase the symbol rate. IIRC there's something similar going on with real human languages too. For instance Chinese usually takes fewer phonemes than Spanish to express the same information. However Spanish speakers tend to speak significantly faster than the Chinese to "make up" for it. The fact that Spanish uses more sound to carry the same amount of informations makes it more resilient to "data loss" and allows a faster speech rate.
- cwyers 9y agoSo, I don't disagree with that. What I disagree with is the characterization of the machine as doing any of that. "[H]ad simply modified human language for the purposes of more efficient interaction" anthropomorphizes the hell out of the machine, and confuses the issue. The machines don't understand English or any other human language and weren't modifying it to any sort of purpose. The machines were given a training corpus and made to communicate between agents and something fell out, and the exact manner of it was about as intentional as an apple falling out of a tree intends to hit this branch or that branch on the way down.
- nothis 9y agoIMO it's also a little condescending to claim readers couldn't tell that this isn't about "robots trying to kill us". The creepy part, the reason the people who actually DO know their shit bothered to ever publish these findings, is that machine learning can lead to code where it's practically impossible for a human being to understand how it gets to an end result. It IS creepy to find an AI developing a language we can no longer follow! Yea, the newspaper illustrations with the Terminator (oh, hey, this article does it, too, but I guess it doesn't count because it's "ironical") are exaggerations but who doesn't get that?
- adityab 9y agoThis stuff sounds funny now, and some of us grad students had a good laugh. But I am worried about the future of ML reporting. The "field" is growing fast and I think we don't have nearly as many science communicators for AI/ML in particular and CS in general, as in other fields. I saw comments by lots of genuinely afraid laypeople who were producing platitudes to the effect that scientists don't have common sense, that we're "playing god"... etc. Also scary stuff things like the need to take action against evil scientists before it's too late. There are genuinely bad things that could come of such reporting. Like knee-jerk regulations being imposed on AI research due to irrational fears, or worse - scared and angry vigilantes going after researchers personally. It's not practical to educate everyone in ML, I wonder how we will solve this problem.
- blowski 9y agoSeems like the same problem as nuclear power, GM foods, and just about every other new but complex technology. People don't understand it, and we always fear what we don't understand.
- adityab 9y agoTrue, but none of those fields have the same kind of "end of humanity" connotations attached, in the general psyche. Clarifications by well-known researchers don't travel as far and wide as urgency-signaling clickbait...
- lotyrin 9y agoNuclear power generation and agricultural GMOs definitely do have their opponents that use end-of-the-world kinds of arguments, and the uninformed that accept those. Nuclear especially would be my go-to pristine example of this effect.
- tgragnato 9y agoThe nuclear has been demonized (in popular culture) due to the psyops of the Cold War. A backslash against nuclear power generation has always been expected. The use of GMOs has been demonized since corporations decided to patent them and sue farmers that owns naturally hybridized plantations. (see agricultural patent trolls) Abuse is what generates criticism and resistance to the use of new technology. Yes, the reasons you hear may not be wholly technically correct. Again, if the trust of internet users is abused, and those who browse are tracked and profiled and spyed on; then should we be surprised that there are publications like this? That's how you get stories about creepy AIs. And a "technically wrong" label doesn't help, preventing abuse is much more effective.
- wonder_bread 9y agoSeems like a tactful play by FB to promote their efforts by letting this 'event' leak out to the press
- nl 9y agoFacebook put blog posts out about the paper and mentioned in passing that they stopped training because they didn't have a method of stopping this happening. Then this stupid interpretation of the events started being thrown around.
- MichaelMoser123 9y agoAnd the press is searching for stories, it's August and lot of newsmakers are out for vacations...
- schoen 9y agoA longstanding tradition: https://en.wikipedia.org/wiki/Silly_season https://en.wikipedia.org/wiki/Silly_season
- gabrielgoh 9y agoA fascinating aspect of this entire kerfuffle is that the meta story, the one about sensationalism and "bad journalism" is too, a form of media sensationalism that plays to the ears of a more sober and skeptical audience who wants news about "media sensationalism" and "AI hype". A bit of digging reveals that no serious news outlet really got this wrong (correct me if I'm wrong!), and most of the sensationalist headlines were from British tabloids https://www.theatlantic.com/technology/archive/2017/06/what-an-ais-non-human-language-actually-looks-like/530934/ https://www.theatlantic.com/technology/archive/2017/06/what-... http://www.mirror.co.uk/tech/robot-intelligence-dangerous-experts-warning-10908711 http://www.mirror.co.uk/tech/robot-intelligence-dangerous-ex... but even more surprisingly, the articles themselves demonstrated a fairly sober understanding of what is going on. The only mistake they made was spinning a mundane story way out of proportion, something tabloids literally every day, and have done since their conception.
- bahjoite 9y agoA minor point: theatlantic.com isn't a British tabloid.
- awkwarddaturtle 9y agoIt is a propaganda outlet now owned by Steve Jobs' wife. https://en.wikipedia.org/wiki/The_Atlantic https://en.wikipedia.org/wiki/The_Atlantic https://en.wikipedia.org/wiki/Emerson_Collective https://en.wikipedia.org/wiki/Emerson_Collective The move by tech industry people into media/propaganda is rather worrying.
- grillvogel 9y agothe bots learned how to lie. how is that not terrifying?
- visarga 9y agoMy /dev/random sometimes outputs poetry. How's that not endearing?
- red75prime 9y agoIt is not goal directed.
- bhhaskin 9y agoDid it learn how to lie though? Did it learn the art of deception and all of the complex concepts behind that? Did it have intent and motive to deceive, Or was it simply a glitch in the underlying logic? The chat bots weren't sentient and could not think for them selves. This is really a story that never should have been, but due to the awful state the media is in it works as a great PR peace for Facebook.
- grillvogel 9y agoyes. >These dialogue rollouts led to bots that negotiated harder and proposed the final deal more often than their counterparts. The bots were also able to produce novel sentences rather than just relying on sentences encountered through training data. And remarkably, the bots engaged in some sly strategizing. There were instances when the bot feigned interest in an item that had no value to them and then pretended to compromise later by conceding it in exchange for something it actually wanted. In a statement, Facebook said, "This behavior was not programmed by the researchers but was discovered by the bot as a method for trying to achieve its goals." https://www.engadget.com/2017/06/14/facebook-bot-lie-better-bargains/ https://www.engadget.com/2017/06/14/facebook-bot-lie-better-...
- imh 9y agoThis has all of my favorite media failings in one! It's like bingo! We have the game of telephone, with outlets reporting on outlets reporting on outlets, each diverging further from the primary source. We have attention grabbing headlines completely removed from the article content. Complete lack of oversight, where if you asked any expert they'd say "this is totally wrong." Big media names lending their platforms to "contributors" providing fake credibility with none of the accountability. All of that going viral despite it being complete nonsense! Yay 2017.
- Jedi72 9y agoMy Mum phoned me last night across 3 time zones because she heard about this and was worried. The people writing these articles are responsible for a lot of fear mongering amongst lay people.
- spraak 9y agoCould someone at Facebook have paid for this article to be written?
- amelius 9y agoWhy don't we just make user-tracking for advertisement purposes illegal? That would solve a lot of problems, because the only reason these companies are interested in our behavior is to influence us.