4 ms·
Won't we enter another AI winter unless we can solve things like finding something better than backpropagation or just jumping to the Student model in a Distill
by skunkworker 7y ago
Won't we enter another AI winter unless we can solve things like finding something better than backpropagation or just jumping to the Student model in a Distillation teacher-student model, letting us know there have to be better means out there that we haven't achieved. Though I could be completely wrong and Hinton could come out tomorrow with a groundbreaking paper that furthers another 5 to 10 years of iterative improvements.
- dijksterhuis 7y agoHinton already came out with a “groundbreaking” paper not too long ago - capsule networks.
- p1esk 7y agoCapsules papers don’t strike me as particularly groundbreaking. Still convolving over every single location in the input image, still using backrop. Vectors instead of scalars for activations and a novel way to pass signals between layers? Sure, why not. But if it wasn’t from Hinton no one would’ve noticed. The initial results were underwhelming, and there has been zero progress even 2 years later.