2 ms·
Fast Feedforward Networks
- goedelwasright 3y agoThis is a really cool idea. By dividing the space into regions, one avoids computing with neurons that do not play any role beside the given region. This could be a game-changer in architectures like MLP-mixer and perhaps even transformers.