3 ms·

that line of research is still going. https://github.com/lucidrains/block-recurrent-transformer-pytorch https://github.com/lucidrains/block-recurrent-transforme

that line of research is still going. https://github.com/lucidrains/block-recurrent-transformer-pytorch https://github.com/lucidrains/block-recurrent-transformer-py... i think it is worth continuing research on both fronts.

1024core 4y ago
Of course, I'm not saying "don't do research". I'm just saying that I don't think this context-length war will lead us to long-term sustainable gains.
- deleted 4y ago
  [deleted]