3 ms·
The authors didn't try very hard to solve the problem (although it's still slightly interesting). Many networks can detect if two never-before-seen objects are
by paulsutter 8y ago
The authors didn't try very hard to solve the problem (although it's still slightly interesting).
Many networks can detect if two never-before-seen objects are the same or different. Simple example is Koch's Siamese Neural Networks[1]. More useful example is Facenet[2], which can tell if two faces are the same or different, even if never trained on these specific faces. Facenet learns an n-dimensional embedding and similarity is estimated by euclidean distance. These are cool papers by the way. The Omniglot dataset[3] used in Koch is a sort of an "inverse MNIST". Omniglot has 20 examples each of 1600 different characters, vs MNIST's 60,000 examples of 10 different characters.
[1] https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf
[2] https://arxiv.org/abs/1503.03832 https://arxiv.org/abs/1503.03832
[3] https://github.com/brendenlake/omniglot https://github.com/brendenlake/omniglot
- gambler 8y agoThis just goes to show how messed up the whole AI field is right now. It shouldn't be about "trying hard". Either they used the right network for their analysis, or they didn't. It should be possible to do negative analysis of AI capabilities. But since there is almost no theory behind this stuff right now (just test sets and architectures targeting particular test sets), anyone can always claim that any negative analysis just didn't target the "right" architecture.
- deleted 8y ago[deleted]
- randcraw 8y agoThis does seem to reduce to one-shot learning, but face dissimilarity metrics won't work well to describe similarity between unstructured objects like the authors propose. Faces have a limited set of common features that are readily separable, quantifiable, and comparable, which is why face recognition can be done rather well just using a SVM. When the set of common features falls to zero, as in this paper, a face-specific net architecture will not do one-shot recognition across independent faceted objects nearly as well.
- paulsutter 8y agoTake a look at the other paper, omniglot is more similar to this challenge
- nmca 8y agoI think you've missed the point slightly. This paper looks at two cases, one of which is the "two images, two embeddings, one operation" (eg siamese-style). They note that this works well. They then go on to note that the problem of "are these two components at aribtrary locations of an otherwise blank image the same or different" is not well solved. This cannot be addressed directly by Siamese nets (well, or at least kind-of-can't. Arguably relational networks are similar to taking pixel pairwise functions, and thus have some relation to Siamese nets.) In any case, they directly exclude the case you mention.
- paulsutter 8y agoMy comment was on the article, which was flat out wrong, and thanks for pointing that out in the paper I had missed that. Their paper was still unimaginative. Trying a handful of vanilla networks doesn’t prove anything. The right framing would have been, “here’s an interesting case to work on” instead of pretending to have identified a basic limitation of neural networks. This is more like a research idea than research.
- nmca 8y agoAh, I missed that you were referring to the article. To be honest, I skipped it and went to the paper :) Apologies!