Research in this area has always been cyclical. The Neural ODE paper that won a best paper award at Neurips two years ago just implements a technique that is co

Research in this area has always been cyclical. The Neural ODE paper that won a best paper award at Neurips two years ago just implements a technique that is common knowledge in the optimal control theory community since the 60s. I wouldn’t dismiss an approach because it isn’t popular right now. Biological neurons force you to think about constraints you normally wouldn’t and crucially have solved temporal credit assignment, memory etc. in interesting ways that already have inspired some breakthroughs. As an example the replay memory idea was directly inspired by the rough understanding neuroscientist developed of the role of the hypocampus in learning.