3 ms·
Cool stuff! How does this compare with Fastchat, which seems like another open source project that helps run LLM models? At a glance, it seems like it's going
by phsource 3y ago
Cool stuff! How does this compare with Fastchat, which seems like another open source project that helps run LLM models?
At a glance, it seems like it's going for lots of similar goals (run LLMs with interoperable APIs):
https://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat
- chaoyu 3y agoOpenLLM in comparison focuses more on building LLM apps for production. For example, the integration with LangChain + BentoML makes it easy to run multiple LLMs in parallel across multiple GPUs/Nodes, or chain LLMs with other type of AI/ML models, and deploy the entire pipeline on Kubernete (via Yatai or BentoCloud). Disclaimer: I helped build BentoML and OpenLLM.