3 ms·
"Important benchmarks" aren't the same as being useful and reliable. I tried using Gemini Pro for tasks that GPT-3.5 does very well on and it fell apart complet
by ianbicking 3y ago
"Important benchmarks" aren't the same as being useful and reliable. I tried using Gemini Pro for tasks that GPT-3.5 does very well on and it fell apart completely. From reading the reviews of people using it, I get the impression Gemini Pro is good at writing marketing copy and other stuff that GPT-3 was good at, and I suspect a lot of benchmarks are over focused on that same text completion modality.
Honestly Gemini Pro is so unimpressive I expect even GPT-3.5 will beat Gemini Ultra in terms of utility, at least for the next 6 months.