19 ms·
I wonder the same too, also is Gemini Pro supposed to beat GPT-4?
by MillionOClock 3y ago
I wonder the same too, also is Gemini Pro supposed to beat GPT-4?
- m3at 3y agoI thought so too but no, Gemini _Ultra_ is supposed to beat GPT-4. The one available through Bard is Gemini Pro. The wording sure is confusing! Looking at the technical report [1], on selected benchmarks, rounded score in %: Dataset | Gemini Ultra | Gemini Pro | GPT-4 MMLU | 90 | 79 | 87 BIG-Bench-Hard | 84 | 75 | 83 HellaSwag | 88 | 85 | 95 Natural2Code | 75 | 70 | 74 WMT23 | 74 | 72 | 74 [1] https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf https://storage.googleapis.com/deepmind-media/gemini/gemini_...