4 ms·

Litellm maintains list of model infos here: https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json https://github.com/BerriAI/litell

Litellm maintains list of model infos here: https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json https://github.com/BerriAI/litellm/blob/main/model_prices_an... Simple UI to search: https://models.litellm.ai/?q=llama3 https://models.litellm.ai/?q=llama3

alextttty 2y ago
Thank you! yeah great source. Do they track throughput for open source models? and inference engines Thats kind of data I want to find as well
- Deathmax 2y ago
  For throughput data, well, you need to actually run prompts to gather the data which racks up costs fast and performance can vary based on input prompt lengths. The two sources I use are OpenRouter's provider breakdown [1] and Unify's runtime benchmarks [2]. [1]: https://openrouter.ai/models/meta-llama/llama-3.1-70b-instruct https://openrouter.ai/models/meta-llama/llama-3.1-70b-instru... [2]: https://unify.ai/benchmarks/llama-3.1-70b-chat https://unify.ai/benchmarks/llama-3.1-70b-chat
CuriouslyC 2y ago
Thanks, this saved me from scraping something.
aurareturn 2y ago
Amazing