Geoff Hinton gave a talk last week at Berkeley on exactly this problem - in pixel space, object identities are all tangled up with location/pose information in

Geoff Hinton gave a talk last week at Berkeley on exactly this problem - in pixel space, object identities are all tangled up with location/pose information in a very nonlinear way; it would be nice to find a representation that actually preserves both components while disentangling them ("equivariance") instead of just throwing away all of the spatial information ("invariance", what convnets do). He's done some work on this, a lot of which is apparently unpublished, but gave a reference to one older paper covering some of the ideas: https://www.cs.toronto.edu/~hinton/absps/transauto6.pdf https://www.cs.toronto.edu/~hinton/absps/transauto6.pdf