3 ms·
A lot of attention window stuff is fluff IMPE, just gotta look at the benchmarks regardless of what the raw numbers sat.
by tbalsam 3y ago
A lot of attention window stuff is fluff IMPE, just gotta look at the benchmarks regardless of what the raw numbers sat.
- ComputerGuru 3y agoBenchmarks are, by definition, artificial. I'm speaking from real-world experience, not from "based off the architecture, here's my guess" and lack of context or poor recall within the "supported" context window is a real problem.
- tbalsam 3y agoYes, this is partially due, among other reasons, to the fact that inference induces a domain shift compared to training due to the use of teacher forcing.