This is eliding most of the history of neural networks as a hot reasearch topics. Since the late 50's early 1960s they've had several resurgences in interest,

This is eliding most of the history of neural networks as a hot reasearch topics. Since the late 50's early 1960s they've had several resurgences in interest, e.g. 80s RNNs, 90s stuff around several centers. Hell people were doing interesting things with them commercially in the 90's. The late 90s and early 00's had a lot of other interest (kernel methods, SVMs) but NN folks kept plugging, and the hardware to hit the next level was just around the corner. The resurgence you're noting are papers with a 10 year tail before them (hell, most of the deep concepts were initiated decades before but lacked both the data sources and efficient hardware to really work them out). This stuff has a long and deeply connected history.