I'm not convinced that a person who's never seen animals before could tell the difference between all future dogs and cats from a single training example. Human

I'm not convinced that a person who's never seen animals before could tell the difference between all future dogs and cats from a single training example. Humans draw upon a lifetime of learning and experience to achieve this 'one shot learning' capability. If you take a pre-trained convnet (which, by analogy is like a person who has had 'life experience' of looking at objects), and extract activations for unseen object categories, in many cases you CAN one-shot-learn these new object categories. Try feeding them into a SVM or use L2 distance between test images and the one-shot exemplar image. On top of this, there's a lot of work on memory-augmented nets and meta-learning for learning new categories on the fly.