At Predibase, we recently conducted 700+ fine-tuning experiments to benchmark the performance of popular open-source LLMs across 30 tasks and compared their res

At Predibase, we recently conducted 700+ fine-tuning experiments to benchmark the performance of popular open-source LLMs across 30 tasks and compared their results to GPT-4. 85% of the time they beat GPT-4. You can see the results here: https://predibase.com/fine-tuning-index https://predibase.com/fine-tuning-index. The site has a series of interactive charts and a link to our Arxiv paper.