2 ms·
What would the context length look like for this setup? How quick would the 6-7 tps degrade once you hit say 20k tokens?
by nexus_six 2y ago
What would the context length look like for this setup? How quick would the 6-7 tps degrade once you hit say 20k tokens?