3 ms·
By the way: I am intrigued by that «I built the scraper by pasting example HTML into Claude» - Simon picked Claude for this specific task. Is anyone maintainin
by mdp2021 2y ago
By the way: I am intrigued by that «I built the scraper by pasting example HTML into Claude» - Simon picked Claude for this specific task.
Is anyone maintaining benchmarked or informal (anecdotal, empirical) lists of what are the strong abilities (and maybe weak ones) per LLM?
- lgas 2y agoI don't know if anyone's doing that but Claude's superiority at coding (currently, amongst non-reasoning models at least) is widely known.
- RSHEPP 2y agoAgreed. Using Claude Sonnet 3.5-7 with Aider produces significantly better output than any other models I have tried.
- simonw 2y agoI've been defaulting to Claude 3.7 Sonnet for a while now as it produces Python and JavaScript that fit my personal tastes, but to be honest I'd expect any of the leading models (GPT-4o/4.5/o1/o3, Gemini 1.5 or 2.0 or 2.5, probably the larger Llamas) to be able to do this well. Dumping a chunk of HTML or JSON directly into a model and saying "write something that converts this to format X" tends to just work in my experience.
- PeterStuer 2y agoBeen using OpenAI for most my api use in production and just started with Anhropic. Having to grind up customer tiers once again realy makes it difficult to experiment as 'Tier 1' hit it's limit after 15 mins. And no, I do not want to 'contact sales' or wait weeks when I'm merely checking out initial viability. Just let me buy and use my available credits.
- rspoerri 2y agoi assume you know the leaderboard, which at least shows the strongest ones in a specific area: https://lmarena.ai/?leaderboard= https://lmarena.ai/?leaderboard=
- mdp2021 2y agoThank you, I did not remember the "Category" selector ("Hard Prompt", "Instruction-Following", "Math"...). Descriptive blog page at https://blog.lmarena.ai/blog/2024/arena-category/ https://blog.lmarena.ai/blog/2024/arena-category/