I didn't see a maths major in your background but I'm surprised you seem to dance around how important back-propagation is, and how different it is from what yo

I didn't see a maths major in your background but I'm surprised you seem to dance around how important back-propagation is, and how different it is from what you could find in nature. The major advances are the implementation details, and I think many would consider the network topology research constrained not by our imagination but by our implementations.