3 ms·
I can answer that from experience (just built a couple Q&A style sites), although a bit subjectively. Compared with the GPTs, Mixtral (and derivatives) perform
by moralestapia 3y ago
I can answer that from experience (just built a couple Q&A style sites), although a bit subjectively.
Compared with the GPTs, Mixtral (and derivatives) perform:
* 5% of the time as good as GPT4
* 75% of the time on par with GPT3.5
* 20% of the time a bit worse than GPT3.5 (too chatty and hallucinates with ease, although one could argue a better prompt could improve things a lot)
Advantages of Mixtral, for me:
* Cost 10-100x cheaper
* Faster completion time
* More deterministic output, in the sense that if you run the same query several times you get the same answers back (but they could be wrong). GPT almost always gives me an answer of great quality BUT with a lot of variance across them; even with temperature set to zero. This is a PITA when you want some sort of predictable, structured output like a JSON object.
- sjwhevvvvvsj 3y agoAnother factor is they monkey around with GPT constantly. The GPT you get on Monday may be totally weird on Friday. Whereas for a local model it doesn’t change unless YOU change it.