3 ms·
Benchmarks are, by definition, artificial. I'm speaking from real-world experience, not from "based off the architecture, here's my guess" and lack of context o
by ComputerGuru 3y ago
Benchmarks are, by definition, artificial. I'm speaking from real-world experience, not from "based off the architecture, here's my guess" and lack of context or poor recall within the "supported" context window is a real problem.
- tbalsam 3y agoYes, this is partially due, among other reasons, to the fact that inference induces a domain shift compared to training due to the use of teacher forcing.