Think of a decision boundary that a neural net makes. And let's assume it's a 3 dimensional surface. That surface is going to have peaks going up and peaks goin

Think of a decision boundary that a neural net makes. And let's assume it's a 3 dimensional surface. That surface is going to have peaks going up and peaks going down. Backprop determines the differential with respect to the error. You can use that to lower the error in classification, but there's nothing preventing you from going the other way. You would normally descend into the valleys, but you can just as well climb the mountain, to find features that are extremely unlikely to be found on an elephant, and with more training you can find stuff that's commonly found on non-elephant things, and train those features on those. What I mean is that if it only sees positive examples, it might conclude that anything means it's an elephant, regardless of what exactly it is. So that's why I have to qualify by saying a "good" feature. Furthermore there are ways to generate good negative examples from the only positive input data. For instance, you can use random static as an input. Given that static (correctly generated) does not have any information content, there's nothing in there that's an elephant, and you can train it away from that. Other obvious ways to generate more data is to do reverse training : determine what input data would make the network say that that is absolutely not an elephant, after seeing a few elephants. Assume it gets that right, that it isn't an elephant. Insert some static to generate more data. Take that input, flip it, rotate it, zoom it, add static, change color distribution and train those as non-elephants. Likewise, take the input pictures. There should be other stuff in there than elephants. So figure out which values trigger the network (not all pixels matter for the end result, in fact, most won't). Find the ones that don't matter for all pictures, and collage them together. Train as non-elephants. Do the same in reverse (find elephant pixels, and use those to paste more elephants in other pictures). You need to do this to keep your data balanced : there should be equal number of positive and negative training generated items. I'm not sure how you could get the adversarial network approach to work for this. At this point, you have still not given it any negative example, but there's lots to train on.