4 ms·
LLMs Leaderboard
- niallmurphy-ie 3y agoMissing WizardLM unfortunately. I was using it yesterday and found it very impressive.
- mdaniel 3y agoDid you see https://github.com/lm-sys/FastChat/blob/main/docs/arena.md#how-to-add-a-new-model https://github.com/lm-sys/FastChat/blob/main/docs/arena.md#h... ?
- tomas789 3y agoNice concept. Similar techniques can be found in neuroevolution, where genetic algorithms optimize neural networks. One major benefit is the ability to compare the performance of both subpar and exceptional models on a relatively fair basis. Considering this evidence, I anticipate that this approach will gain greater significance as the quality of language models surpasses our existing benchmarks. However, it's worth noting that a potential pitfall arises when certain models outshine the rest of the group.
- hackernewds 3y agowhere is Bard?
- oidar 3y agoI don't think Bard has an API available - so it can't be fairly tested.
- qeternity 3y agoNot suggesting it’s incorrect but the people who built this list also rank their own model higher than all other llama derivatives.
- YetAnotherNick 3y agoIt's blind test, and the code to compute scores is open. So unless they are deliberately modifying the data it should be a fair ranking. Just I think openassistant llama(30B) version is better than vicuna 13b, but it is not included in the test.
- adt 3y agoHere's my comprehensive version (123 LLMs to PaLM 2 @ 10/May/2023), without the 'ranking' part: https://lifearchitect.ai/models-table/ https://lifearchitect.ai/models-table/
- jadbox 3y agoDiligent work aggregating this together. I'm curious what started you on this journey to keep this list going?
- ranguna 3y agoInteresting benchmark! Would mosaic mpt be useful to benchmark as well?