3 ms·
And there have been a lot of approaches to do this, my favorite one being the idea that maybe if we just randomly zap out some of the neurons while we train th
by crdrost 3y ago
And there have been a lot of approaches to do this, my favorite one being the idea that maybe if we just randomly zap out some of the neurons while we train the rest, that forcing it to acquire that redundancy might privilege structured representations over memorization. Just always seemed like some fraternity prank, “if you REALLY know the tenets of Delta Mu Beta you can recite them when drunk after we spin you around in a circle twelve times fast!”
- whimsicalism 3y agohttps://nitter.net/Yampeleg/status/1688441683946377216 https://nitter.net/Yampeleg/status/1688441683946377216
- two_in_one 3y ago> just randomly zap out some of the neurons while we train the rest It's already done: https://pytorch.org/docs/stable/generated/torch.nn.functional.dropout.html https://pytorch.org/docs/stable/generated/torch.nn.functiona...