14 ms·
Mistral 7b is serviceable for short contexts. If you have a longer conversation, token generation can start to lag a lot.
by all2 3y ago
Mistral 7b is serviceable for short contexts. If you have a longer conversation, token generation can start to lag a lot.