3 ms·
My experience in "Applied Research" is that often "good ideas from lower ranked researchers" (or good ideas from anyone really) is "I saw this cool paper, let's
by thomasahle 3y ago
My experience in "Applied Research" is that often "good ideas from lower ranked researchers" (or good ideas from anyone really) is "I saw this cool paper, let's try and implement that". That doesn't mean top people like Hinton should get all the credit, but let's not kid ourselves and believe most of the ideas didn't origin in academia.
One of GOpenAI's recent breakthroughs was switching to FlashAttention, invented at Stanford and University at Buffalo.
- lostmsu 3y agoIsn't Hinton's paper on backprop just a "let's try to implement that" for a multilayer network?
- thomasahle 3y agoI'm not going to claim Hinton "invented backprop". He even says he didn't himself: https://www.reddit.com/r/MachineLearning/comments/g5ali0/comment/fo8rew9/ https://www.reddit.com/r/MachineLearning/comments/g5ali0/com...