5 ms·

I don't know of any 180B models that outperform gpt-3.5, including Falcon (according to benchmarks). There are fine-tuned 70B models have higher benchmark scor

I don't know of any 180B models that outperform gpt-3.5, including Falcon (according to benchmarks). There are fine-tuned 70B models have higher benchmark scores than gpt-3.5, and when quantized they can run on a 64GB Macbook. https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/discussions/30 https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb... https://huggingface.co/spaces/gsaivinay/open_llm_leaderboard https://huggingface.co/spaces/gsaivinay/open_llm_leaderboard

deleted 3y ago
[deleted]
Der_Einzige 3y ago
Note that quantization has serious impact on the models capabilities.
selfhoster11 3y ago
Benchmarks don't necessarily reflect real-world performance. Especially given that they poorly measure more esoteric aspects of the model that, for now, can only be judged qualitatively. I would wait for a bit to see what the community comes up with before writing off Falcon-180B.