2 ms·
Yeah, Haiku 3.5 doesn't really count as a budget model - Anthropic have kept the original Claude 3 Haiku around for their budget entry. I collected notes on th
by simonw 2y ago
Yeah, Haiku 3.5 doesn't really count as a budget model - Anthropic have kept the original Claude 3 Haiku around for their budget entry.
I collected notes on the lowest cost hosted LLMs from the major vendors when I wrote up Amazon Nova last week: https://simonwillison.net/2024/Dec/4/amazon-nova/ https://simonwillison.net/2024/Dec/4/amazon-nova/
Nova Micro is $0.035/$0.14 and Google's Gemini 1.5 Flash 8B is $0.0375/$0.15 - just beating those OpenRouter prices, but it may well be that the Qwen models provide better results.
- mythz 2y agoYeah I'm currently using Gemini 2 Flash (exp) free quota for a premium hosted model, it's a surprisingly great model, IMO Google has caught up with the leaders with their latest experimental models. I've also tested Nova's models, which are pretty high quality and exception value (lite/micro) for their performance. Also worth shouting out you can get Meta's latest llama-3.3:70b (comparable to llama3.1:405b but must faster and cheaper) within GroqCloud's free quotas running at an impressive 276 tok/s.
- sharih 2y agoGroq limits context window to 8192 is that your experience too
- antupis 2y agoHave you tested tool calling capabilities of these cheaper models?
- simonw 2y agoNot yet. A lot of them make bold claims and there are benchmarks like https://gorilla.cs.berkeley.edu/leaderboard.html https://gorilla.cs.berkeley.edu/leaderboard.html but I don't have my own function calling test harness setup yet.
- leobg 2y agoYou know that you can make any model call and use tools simply by giving it few shot examples and writing your own parsing logic. I’ve done it many times for clients, both at the prompt and at the fine-tune level.