4 ms·
The ALiBi paper shows that our method beats the sinusoidal PE you refer to across many benchmarks. https://arxiv.org/abs/2108.12409 https://arxiv.org/abs/2108.1
by ofirpress 3y ago
The ALiBi paper shows that our method beats the sinusoidal PE you refer to across many benchmarks. https://arxiv.org/abs/2108.12409 https://arxiv.org/abs/2108.12409