3 ms·
13B models probably cannot directly compare with ChatGPT 4 which maybe +1T parameters or a 5 way MoE of 200B each - or something like that. So you can not like
by bhouston 3y ago
13B models probably cannot directly compare with ChatGPT 4 which maybe +1T parameters or a 5 way MoE of 200B each - or something like that. So you can not likely run a model competitive with ChatGPT locally in the near term.
- tombert 3y agoI have a server with a bunch of PCIe slots and like 4 Nvidia GPUs with 24GB of RAM each. What's the best model I can realistically run?
- bhouston 3y agoHere are some scorecards: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb... https://paperswithcode.com/sota/sentence-completion-on-hellaswag https://paperswithcode.com/sota/sentence-completion-on-hella... https://paperswithcode.com/sota/common-sense-reasoning-on-winogrande https://paperswithcode.com/sota/common-sense-reasoning-on-wi... https://paperswithcode.com/sota/common-sense-reasoning-on-arc-challenge https://paperswithcode.com/sota/common-sense-reasoning-on-ar... https://paperswithcode.com/sota/common-sense-reasoning-on-commonsenseqa https://paperswithcode.com/sota/common-sense-reasoning-on-co...