weichiang

4 ms·

1.
by weichiang 3y ago
say using A10G ~$1.2/hr and with full utilization on vllm 112 reqs/min => per req ~$0.00018 versus gpt-3.5 turbo $0.002 per 1k token
2.
by weichiang 3y ago
the cost would be depending on GPU type/serving system/traffic pattern. check out some throughput comparison from vllm's blog post https://vllm.ai/ if you serve 7B on cost-optimized GPUs (A10G/L4) and ke
3.
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena(twitter.com)
20 points by weichiang 3y ago|0 comments
4.
Google PaLM 2 ranked 6th on the LLM benchmark in the wild(twitter.com)
1 points by weichiang 3y ago|0 comments
5.
Chatbot Arena: a crowd-sourced LLM leaderboard(twitter.com)
1 points by weichiang 3y ago|1 comments
6.
by weichiang 3y ago
leaderboard link: https://chat.lmsys.org/?leaderboard
7.
by weichiang 3y ago
Surprised to learn StableLM is worse than plain LLaMA. link to their leaderboard: leaderboard.lmsys.org
8.
by weichiang 3y ago
I asked GPT4All one of Vicuna's benchmark questions: "What if the Internet had been invented during the Renaissance period?" Check out their responses: https://imgur.com/a/mPrdZ1W More questions here: h
9.
State-of-the-Art Chatbot, Vicuna-7B, now runs on MacBook with GPU acceleration(twitter.com)
126 points by weichiang 3y ago|84 comments
10.
State-of-the-art open-source chatbot, Vicuna-13B, just released model weights(twitter.com)
271 points by weichiang 3y ago|139 comments
11.
by weichiang 3y ago
See the original Vicuna post: https://news.ycombinator.com/item?id=35378683
12.
Who's GPT-4's favorite? Battles between state-of-the-art chatbots.(vicuna.lmsys.org)
6 points by weichiang 3y ago|4 comments
13.
by weichiang 4y ago
just tried it worked okay for me tho. > Can you name all of the soccer players whose names contain a Z > Here are all the soccer players whose names contain a "Z" that I could find: Zlatan Ibrahimovic Zinedine Zidane Zlatko