4 ms·
This is better than most articles I've seen on this subject, but it still falls victim to AI hype. > Although some reports insinuate that the bots had at this
by cwyers 9y ago
This is better than most articles I've seen on this subject, but it still falls victim to AI hype.
> Although some reports insinuate that the bots had at this point invented a new language in order to elude their human masters, a better explanation is that the neural networks had simply modified human language for the purposes of more efficient interaction.
From the Gizmodo article it links to (which is also okay, but not great):
> “Agents will drift off understandable language and invent codewords for themselves,” FAIR visiting researcher Dhruv Batra said. “Like if I say ‘the’ five times, you interpret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.”
Except... this isn't shorthand. "I want five balls" is shorter than "Give me balls the the the the the," or whatever else it could come up with. It's not more efficient. It's just... dumb. The bots don't actually understand the words the way humans do. Because they don't understand words and language, they're using the words as tokens. It's this primitive level of communication that just happens to have words assigned to it. Even the calmer takes on this are anthromorphizing the machines too much and attributing intelligence to the complete lack thereof.
- simias 9y agoIt depends what you're optimizing for I guess. If you want to reduce the length of the sequence then "Give me balls the the the the the" is silly but if the point is to reduce the vocabulary then it makes sense. After all most machines currently communicate by exchanging sequences of exactly two symbols, 0 and 1, not some complex phonology. Maybe when bandwidth is not as constrained as human speech it's actually more efficient to reduce the vocabulary and increase the symbol rate. IIRC there's something similar going on with real human languages too. For instance Chinese usually takes fewer phonemes than Spanish to express the same information. However Spanish speakers tend to speak significantly faster than the Chinese to "make up" for it. The fact that Spanish uses more sound to carry the same amount of informations makes it more resilient to "data loss" and allows a faster speech rate.
- cwyers 9y agoSo, I don't disagree with that. What I disagree with is the characterization of the machine as doing any of that. "[H]ad simply modified human language for the purposes of more efficient interaction" anthropomorphizes the hell out of the machine, and confuses the issue. The machines don't understand English or any other human language and weren't modifying it to any sort of purpose. The machines were given a training corpus and made to communicate between agents and something fell out, and the exact manner of it was about as intentional as an apple falling out of a tree intends to hit this branch or that branch on the way down.
- lemondrops 9y ago> Even the calmer takes on this are anthromorphizing the machines too much and attributing intelligence to the complete lack thereof. Exactly. I haven't read the details of the implementation of the system mentioned in the article, but the outputs remind me a lot of what was returned by a text generating neural network of this tutorial that I did once: http://machinelearningmastery.com/text-generation-lstm-recurrent-neural-networks-python-keras/ http://machinelearningmastery.com/text-generation-lstm-recur... Especially with fewer epochs (<10) the generated text was part gibberish, part endless repetitions of common phrases or basic words like "the" - simply because (surprise!) "the" is one of the more frequently used words in speech. Pulling this out of context, one could also say "This AI is inventing its own language, just by reading Alice in Wonderland!", which is of course utter bullshit.