4 ms·

tried the 7b, it switched to chinese mid-response

• popinman322 2y ago
  Assuming you're doing local inference, have you tried setting a token filter on the model?