3 ms·

Is ALiBi still the sota for this setting, or have there been advances beyond this in the last 8 months? I know there has been a lot of interest in longer contex

Is ALiBi still the sota for this setting, or have there been advances beyond this in the last 8 months? I know there has been a lot of interest in longer context lengths recently.

ipsum2 3y ago
xpos is SoTA right now: https://arxiv.org/pdf/2212.10554.pdf https://arxiv.org/pdf/2212.10554.pdf
- Eridrus 3y ago
  Thanks!