3 ms·
Is there a benchmark comparing the two that I missed? Edit: The huggingface page has 0-shot benchmarks which you can compare against the llama paper https://h
by gpm 4y ago
Is there a benchmark comparing the two that I missed?
Edit: The huggingface page has 0-shot benchmarks which you can compare against the llama paper
https://huggingface.co/cerebras/Cerebras-GPT-13B https://huggingface.co/cerebras/Cerebras-GPT-13B
https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf
- freeqaz 4y agoI'm on mobile and struggled to compare these two tables properly. Would you mind posting a summary of your findings? Here are some values but I don't know what they mean. LLama 60B on the left, Cerebras 13B on the right. PiQA: 82.8 / 76.6 WinoGrade: 77.0 / 64.6 ARC-e: 78.9 / 71.4
- gpm 4y agoReally short summary: LLaMa is better, even smaller LLaMa models. Table format: Benchmark, Cerebras 13B, LLama 7B, LLama 13B, LLama 60B HellaSwag, 51.3, 76.1, 79.2, 84.2 Piqa, 76.6, 79.8, 80.1, 82.8 Wino-Grande, 64.6, 70.1, 73.0, 77.0 Arc-e, 71.4, 72.8, 74.8, 78.9 Arc-c, 36.7, 47.6, 52.7, 56.0 OpenBookQA, 28.6, 57.2, 56.4, 60.2