45 ms·
How do you deal with the 'too many tokens' problem? Like, For 13b, max tokens I've gotten it to is 508 - is there a way to increase the number it will accept?
by MPSimmons 3y ago
How do you deal with the 'too many tokens' problem? Like, For 13b, max tokens I've gotten it to is 508 - is there a way to increase the number it will accept?
- MPSimmons 3y agoOh, I figured this out. It runs with 512 as the default context, but you can increase it with -c