3 ms·
I'm on mobile and struggled to compare these two tables properly. Would you mind posting a summary of your findings? Here are some values but I don't know what
by freeqaz 4y ago
I'm on mobile and struggled to compare these two tables properly. Would you mind posting a summary of your findings?
Here are some values but I don't know what they mean. LLama 60B on the left, Cerebras 13B on the right.
PiQA: 82.8 / 76.6
WinoGrade: 77.0 / 64.6
ARC-e: 78.9 / 71.4
- gpm 4y agoReally short summary: LLaMa is better, even smaller LLaMa models. Table format: Benchmark, Cerebras 13B, LLama 7B, LLama 13B, LLama 60B HellaSwag, 51.3, 76.1, 79.2, 84.2 Piqa, 76.6, 79.8, 80.1, 82.8 Wino-Grande, 64.6, 70.1, 73.0, 77.0 Arc-e, 71.4, 72.8, 74.8, 78.9 Arc-c, 36.7, 47.6, 52.7, 56.0 OpenBookQA, 28.6, 57.2, 56.4, 60.2