11 ms·
Machine Learning Confronts the Elephant in the Room
- lolc 8y agoSomehow the right image where according to the description an elephant was introduced is now identical to the left photo in the article.
- kapauldo 8y agoIs that diagram right? The neural net does the segmentation? There's no pre-process, it literally feeds pixels in?
- deltron3030 8y agoCan't they implement a "virtual representation" or sort of ideal or archetypical scene in an AIs memory that's quickly accessible, and then use that to diff "new unexpected stuff" and refocus on that difference somehow?
- dmichulke 8y agoIf we had such a representation and the mapping from real world (images) to it, we'd basically solved all image recognition tasks because the above is the definition of an image recognition task
- deltron3030 8y agoIn the real world you wouldn't need to know the exact shape of an object to determine what it is, unique features could be enough. For an elephant this could be just a range of skin colors that are unique to elephants. The problem with videos and images vs. the real world is the fidelity, you can't trust images or videos that are previously captured through other equipment. This barrier wouldn't exist if the AI can "see" into the real world, the skin colors of animals can be trusted there!
- doombolt 8y agoWrong. They can do a second take. You should just code it in.
- majos 8y agoIs this sarcasm? The point of the second take is that humans often know they're confused and will go back to think about the image and remedy the confusion, whereas having a neural network just look at the image again isn't going to do anything (plus, existing architectures don't seem to have any capacity to say "I'm confused" anyway).
- antpls 8y agoParent is not sarcastic. Automated recognition systems are systems, which means they are made of several components (sensors, databases, hardwares, softwares) working together. Neural networks are one of those sub-components and no one ever claimed that neural networks are 100% accurate and sufficient to build automated decision systems. The problem described in the article is taken into account when building systems (like autonomous cars) using neural networks.
- sorokod 8y agoThe problem described in the article is taken into account when building systems (like autonomous cars) using neural networks. Where can I find more information on this?
- antpls 8y agoThis is a vast engineering field studied before the existence of neural networks. Keywords are "control theory", "sensor fusion", "automated decision under uncertainty" that you can look on Google, Wikipedia and arXiv. Also "Simultaneous localization and mapping" which is a good example of using uncertain data points from different sensors to build a representation of the reality. In those systems, a neural network is just another sensor providing augmented information.
- sorokod 8y agoIs this not a whole new attack surface?
- TheArcane 8y agoNot really. In addition to object classes, object detection networks learn to ID teh context of an object's presence. In almost every scenario, this context is a desirable learning trait.
- sorokod 8y agoHow about a scenario where an image of an elephant ( or any other out of context example ) is projected on purpose with intent to cause damage?
- QML 8y agoAdversarially generated inputs have been known for some time [1], so I wouldn’t call it a new class of attacks unless 1. there are other and older ways of attacking neural networks 2. new as in the last couple of years. [1] https://blog.openai.com/adversarial-example-research/ https://blog.openai.com/adversarial-example-research/
- p1esk 8y agoYes, this can be used for generating adversarial inputs, and yes, this is a new attack surface. From the paper: The images generated here could be viewed as a variant of adversarial examples, in which small image perturbations (imperceptible to humans) cause a large shift in the network’s output. The images we generate are of a somewhat opposite flavor: while we do not limit the magnitude of the difference between the original and modified image, the detectors are sometimes “blind” to the inserted object. In addition, our examples are not “targeted” in the sense that no optimization process is required to generate them; they seem prevalent enough so that a simple scan of transplanting translated versions of one object in the other can give rise to multiple wrong interpretations.
- antpls 8y agoThe title is a bit misleading / clickbait. It should be "Neural networks Confronts...". Machine Learning isn't all about neutral networks and deep learning. As another comment said, "second take" is not what neural networks are made for. Neural networks are a building block of more complex decision systems, where the weakness of the neural networks are taken into account before automatically commiting to decisions. Otherwise, I guess the article is good at pointing the current limitation of neural networks alone.
- LeanderK 8y agoi wouldn't be so critical. It's journalism and I would intrepret the title more as "Machine Learning-community confronts...". And while ML isn't all about NNs and deep learning, we're quite enticed by them right now and try to push them to see where their limitations are. So I think with a bit more generous interpretation the title is acceptable. I think it's certainly not clickbait, since NNs are right now a very significant part of the ML-research landscape.
- antpls 8y agoWell, to me, this is not journalism. Journalists have a strong ethic and respect for the meaning of words to relay high quality information. This is science vulgarisation, or a magazine article. Not that there is no value in it, but it's an over-generalization.
- dpwm 8y ago> It should be "Neural networks Confronts...". I disagree, for the following reasons: - The paper is called The Elephant in the Room. Technically the paper confronted the popular machine learning approaches, so the reversal would have been a more eye catching headline. "The Elephant in the Room confronts Machine Learning" is a rare opportunity to honestly play with word order in a way that would arguably be more attention-grabbing. - Machine learning is more general than Convolutional Neural Networks, but less general than AI. This seems appropriate for this type of popular science publication. - There are approaches that use attention and RNNs and they are an interesting approach, but by no means mainstream nor producing state-of-the-art classification accuracies. Actually, second take is exactly what an attention-based RNN can do. - The last time I checked the state-of-the-art solutions to image classification involve Convolutional Neural Networks. If there was an approach that was even close to approaching ConvNet performance, I would agree that it is perhaps unfair to condemn the entire field. Even then, we would be in the territory of generalizations in headlines, which are often fair. "{party} proposing legislation" is a probably bigger generalization of headline that would generally pass without criticism.
- weregiraffe 8y agoWhy can't you just train the network on artificially altered photos, with "elephants" randomly scattered around, until it is robust to them?
- deleted 8y ago[deleted]
- throwaway2048 8y agoAre you going to repeat this with every object that exists, or could exist?
- olodus 8y agoNo, you see neural networks are specially afraid of elephants. Just like elephants are especially afraid of mice. And so the circle continues, all the way around to humans who are especially afraid of neural networks and what they will do to society.
- weregiraffe 8y agoI am sorry to inform you that this post is not that funny.
- weregiraffe 8y agoWhat I'd do is train it on a large diverse set of foreign objects, then test it with a different diverse set of foreign objects, to see if it is possible to make it generally robust. Basically, the goal is that any feature which is not overlapping with the "elephant" should not be influenced by it.
- YeGoblynQueenne 8y agoWell, CNNs are already trained on "a large diverse set" of objects, in many different configurations. By the way, something that is not described in the article- the referenced paper also describes experiments where objects where translated inside the source image itself - so not in another image. And the tested networks were still confused. So it's not "foreign" objects that are the problem. It's that neural nets generalise very poorly. Show them something they haven't seen before and their ability to recognise it falls off a cliff.
- calhoun137 8y agoIn this article, learn the secret AI researchers dont want you to know, my non tech friends love talking about it! Gonna have to agree with everyone else here, the title is clickbait and you can just code around it. There is nothing fatal here its just one more problem to solve. Havent we heard all about this issue of image recognition being trickable 1000 times, why is this the top post?
- ag71668 8y agoYeah, rubbish clickbait article which at best misleads uniformed readers.
- ivan_gammel 8y agoPlease, correct me if I'm wrong, but isn't the whole machine learning testing approach based on the recognition the whole image at once in full resolution in one attempt? If so, it's not how animals see and observe the world and this could be a key difference between AI and living beings. What if an algorithm actually had a focused sight? Look at the brightest feature, blur the rest, then try to find another distinctive details, look closer into the parts of the image until the most of it becomes clear. Let other neural network control the scanning process and loop until it develops some confidence in the result. Can it work that way?
- sp332 8y agoYes, check out "attention" mechanisms. The network can use wide convolutions for the image as a whole, effectively mixing data kind of like a blur function, and also use finer convolutions for a small piece of the image that it chooses to focus on.
- ivan_gammel 8y agoDoes the problem of elephants still exist with such approach? It doesn't seem right that an unexpected feature in a corner of the image would have a dramatic impact on the recognition of the features elsewhere.
- sp332 8y agoWell if there's a thing/elephant/chair floating over someone's head in one part of the room, that does change the context for interpreting other things in the room.
- CodesInChaos 8y agoI don't see any evidence of "dramatic" impact. The objects visible in both images only show minor changes in the confidence score . The objects missing in one image are close to the visualization threshold (50% confidence) in the other. And for the "chair turning into a couch", we don't see the likelihood of the chair, because it's hidden behind the box for the couch. So the dramatic changes seen in the image seem to be artifacts of the visualization and not of the network's performance.
- hydrox24 8y agoI know very little about machine vision, so forgive the naïvete of this question: > an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance. Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around? If so, why can't we just call this a computation difficulty problem, simply requiring a way to go back and spend more effort later when required? A problem like this would simply require better computers, and patience. In other words, Is this problem as deeply rooted as the article suggests? Or is it simply a problem with the popular approach to machine vision?
- useful 8y agoEach is harder: * return a result * return a list of results * return a list of results with a confidence weightins * return a list of results with multiple weightings (confidence and complexity) how many if statements would you need to write for each if this was applied to a 3x3 grid like tic-tac-toe? how would you know you are correct?
- ht85 8y agoI think the article is referring to a scene that would be unlikely, not make sense given the context. Like identify a grey mass in the middle of the street as en elephant. "Confusing" would be a poor choice of word here though.
- skierscott 8y ago> a low confidence score Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1]. This happens as the final operation of a neural net, and is required for training. Softmax is a tool to make an array of positive numbers look like a probability distribution: out = x / x.sum() x[i] is a class prediction, but x.sum() != 1. Say if the network was uncertain, x[cat, dog] = [0.03, 0.01]. These are small values that do not imply great confidence (the network was trained on vectors with out.sum() = 1. The network would predict “dog” using softmax because out[dog] = 0.75 > 0.25 = out[cat]. But then in inference/prediction, the confidence is ignored. What if x.sum() is small? That would imply that the network is uncertain. [1]: https://en.m.wikipedia.org/wiki/Softmax_function https://en.m.wikipedia.org/wiki/Softmax_function
- kakarot 8y agoI'm confused about what's so special here. This is analogous to a person from the early days of photography having no knowledge of the possibility of image doctoring, and us feeling "smug" because a photo like this would elicit confusion. It seems to me you just have to run your NN on pairs of doctored and undoctored images, and create a separation of concern between scene continuity and object recognition. It's likely that a lot of implementations currently rely too much on surrounding context for successful categorization. It's also worth exploring certain image analysis techniques during pre-processing to draw out certain attenuated aspects. Edge detection is par for the course, but things like shading are also important to attenuate so that the NN has an easier time learning. If we're dealing with multiple frames and not just a single image, this opens up a whole dimension of temporal analysis that should make it trivial to separate the background from juxtaposed images.
- PurpleBoxDragon 8y ago>It seems to me you just have to run your NN on pairs of doctored and undoctored images, and create a separation of concern between scene continuity and object recognition. This may fix this problem, but what other visual issues will we uncover that might be fixable with the right kind of training but which we haven't yet accounted for. For example, we see so many optical illusions because of how our eyes and brain (and the neurons between them) work. These are often times caused by our sensation or perception incorrectly picking up data, but on purpose (well, as much purpose as an evolved system could have). For example, how neurons in the eye exaggerate differences in brightness or how we are overly sensitive to perceiving faces so that we see them even where they don't exist. What impact does the lack of these have on an artificial vision system? For starters, it would likely indicate the system wouldn't recognize the optical illusion unless trained for it, but are there any other impacts that we need to take into consideration, especially when considering having human lives depending upon the system?
- dukoid 8y agoWouldn't humans who haven't seen Elephants before or who are not trained to recognize the general category of mammals / animals be prone to make similar errors?
- sgt101 8y agoI don't think so - I think that the human may fail to find the correct label for the Elephant (although if they have some knowledge about Elephants they might), but the machine fails to see it - to identify it as a thing, and also it changes the labels it gives other things; so cats become dogs, mugs become books and so on. Humans are not prone to this.
- PurpleBoxDragon 8y ago>who are not trained to recognize the general category of mammals / animals Does such a human exist? Other than perhaps someone blind from birth who had their sight restored, I don't think you could fine such a person give how conditioned we are to recognize animals, perhaps even on a biological level. Closest I can think of is animals that are camouflaged, and I think we would see many humans making mistakes on those. I've definitely have, especially with regards to insects that look like plants. But those animals specifically evolved to throw off predator detection systems.
- p1esk 8y agoIn this case the object detector has been trained to recognize elephants. It's just never seen elephants in such a setting (living room). It would be similar to a situation where a human, who sees an improbable object for a given environment, reevaluates his initial perception/understanding of that environment. I can't come up with a good example of such a situation, but you get the idea.
- vinayms 8y agoI am not an expert in AI/ML, just a casual observer, but I didn't like the tone of the article. Not only it fixates on the speed of processing, it also seems smug and acts as if we know how human brain sees and processes images, while all we have is a bunch of conjectures. That was distracting.
- mcguire 8y agoWell, we do know that the brain does something that CV systems don't, and that it was a part of a recent self-driving car accident.
- mcguire 8y agoThis is an important, and worrying, problem, but I'd be more impressed if the elephant wasn't added as a bad photoshop. That kind of manipulation is going to screw up a lot of the processing.
- PavlikPaja 8y agoFirst, the chair wasn't replaced with a couch - you can see there is yet another rectangle just a few pixels to the left, that likely says "chair". Second, even many people are surprisingly bad at decoding incongruous scenes, which is why hidden object games are a thing.
- CodesInChaos 8y agoThe disappearing cup and book are caused by a similar issue. They "only show detection results with confidence value that exceeds 0.5". The cup is shown at 50% in the original image, so it "disappears" if the presence of the elephant lowers its score even slightly. Same of disappearing books. Presumably the couch showed up because it rose from <0.5 to 0.57. They don't tell us what the score is for the chair, it might even be higher than the score for the couch, since their visualization clearly doesn't sort by probability.
- ickler9 8y agoAgreed. Also, what exact architecture were they using, how were they segmenting, etc. This feels like a rehashing of known challenges using a random implementation struggle to make some broad "AI" judgement.
- crimsonalucard 8y agoI'm aware of the vision algorithm in my head to a certain extent and I'm not sure if machine vision does the same. You can run simple thought experiments to see what your brain actually does to analyze an image. First of all when I look at a scene I am 100% aware of geometry. Irregardless of meaning, words and symbols I can trace out the three dimensional shape of things without associations to words. How do I know I can do this? Simple. Every scene I look at I can basically translate or imagine that scene in my head as wireframe scene or some low poly scene as if it was generated by a computer. Similar to if I look at wireframe scene generated by a computer, my mind can translate it into a scene that looks real. Try it, you can do it. Second, I can look at an actual low poly wireframe model of an elephant and associate it with the word 'elephant.' I do not need color, or detail to know it's an elephant. In fact, with just color and detail alone it is harder for me to identify an elephant. For example if someone takes many very closeup photographs of parts of an elephant like its eye, skin, ear, etc.. and asks me to guess the subject by interpreting the pictures... I become fully aware that I would be accessing a slower, different part of my brain to deduce the meaning. This is a stark contrast to the instantaneous word association established when I look at a wireframe model of an elephant. The speed difference between both ways of identifying an elephant indicate to me that geometric interpretation is the primary driver behind our visual analysis and details like color or texture are tertiary when it comes to the identification of an elephant. I believe the visual cortex determines shape first, then subsequently determines word from shape. If you feed a white sculpture of an elephant or a wireframe of an elephant into one of these deep learning networks it is unlikely you will get the word 'elephant' as output. But if you feed it a real picture of an elephant it can correctly identify the elephant (assuming it was trained against photos of an elephant). Because the delta between a white sculpture of an elephant and an actual picture of an elephant is just color and detail this indicates to me that when you train these deep learning networks to recognize an elephant you are training the network to recognize details. It's a form of over fitting, the training is not general enough to catch geometry. It is correlating blobs of pixels , color and detail with an elephant rather then associating a three dimensional model of it to the word... the opposite of what humans do. In fact I bet you that if you took those very closeup photographs of an elephant and fed it into the network it'd do a better job at recognition versus the picture of a white sculpture of an elephant. This indicates to me that to improve our vision algorithms, the algorithm must first associate pixels with geometry then identify the associated word to the geometry rather than try to associate blobs of pixels to words. Train geometry recognition before word association. My guess is that our minds have specific and genetically determined built in geometry recognition algorithms honed to turn a 2d image into a 3d shape. We do not learn to translate 2d to 3d we are born with that ability hardwired. Where learning comes in is the translation of this shape to a word. Whereas most of the machine learning we focus on in research is image recognition, I believe the brain is actually learning shape and geometry recognition.
- fizixer 8y ago> A visual prank exposes an Achilles’ heel of computer vision systems: Unlike humans, they can’t do a double take. Thanks for giving us ML researchers a TODO. We'll get to work right away.