3 ms·
Are the benchmarks comparing just the models while keeping the harness the same (the open-source Toast harness)?
by docheinestages 2mo ago
Are the benchmarks comparing just the models while keeping the harness the same (the open-source Toast harness)?
- breadislove 2mo agoyes for the retrieval benchmarks. For officeqa pro v2 we used Codex (as databricks did) and for Harvey LAB we used the vanilla harvey benchmark. For these benchmarks we added minimal tools to use mixedbread search and toast 1.