4 ms·

Try some of the Quen models. They have some that are slightly larger than 8b that will fit on your 24gb quite nicely. They have been amazing so far.

• beoberha 2y ago
  Wanted to circle back. The 2.5 coder 7b instruct is amazing, holy crap. It classifies my cases almost perfectly and almost always returns perfectly formatted JSON.
  • Tostino 2y ago
    Yeah, i've been seriously impressed. Glad it worked for your use cases.