3 ms·
The transformer model in GPT-3 has a short context window and no recurrence. Without some significant architecture changes that is a fundamental limit on the pr
by Tarean 6y ago
The transformer model in GPT-3 has a short context window and no recurrence. Without some significant architecture changes that is a fundamental limit on the problems GPT-3 can solve.
- lucidrains 6y agohttps://arxiv.org/abs/1807.03819 https://arxiv.org/abs/1807.03819