2 ms·
Actually there are alternatives by the hundreds, with similar results. Reformer, Linformer, Performer, Longformer... none is better than vanilla overall, they a
by visarga 3y ago
Actually there are alternatives by the hundreds, with similar results. Reformer, Linformer, Performer, Longformer... none is better than vanilla overall, they all have an edge in some use case.
And then we have MLP-mixer which just doesn't do "attention" at all, MLP is all you need. A good solution for edge models.
- homarp 3y agoMLP-mixer: https://news.ycombinator.com/item?id=28581570 https://news.ycombinator.com/item?id=28581570 and https://towardsdatascience.com/mlp-mixer-in-a-nutshell-eccffb68e4fc https://towardsdatascience.com/mlp-mixer-in-a-nutshell-eccff...