I haven't had the opportunity to dive deeply into capsule networks yet, but I have been fairly excited by what little I've read so far. I think that Hinton et a

I haven't had the opportunity to dive deeply into capsule networks yet, but I have been fairly excited by what little I've read so far. I think that Hinton et al are brilliant and definitely taking neural nets in the right direction by trying to approximate more of the mid-level organization of the brain, but I feel like recent attempts to integrate things like attention and memory into nns are still at a very elementary level. This isn't to denigrate recent work at all: this is a stage we have to cross on our way to more powerful intelligent systems, and you're right that pragmatically one of the best ways to further the field is to focus on making models that are useful in their current state even if they poorly approximate more complex systems. On the whole, I'd wager the state of the field is still at the Bohr model level, but progressing rapidly.