38 ms·
How do you reliably compare it with the GPT-4 released in March 2023?
by MichaelZuo 2y ago
How do you reliably compare it with the GPT-4 released in March 2023?
- simonw 2y agoVibes, based on what I can remember using that model for. There's still a gpt-4 model available via the OpenAI API, but it's gpt-4-0613 from June 2023 - the March 2023 snapshot gpt-4-0314 is no longer available. I ran one of my test prompts against that old June 2023 GPT-4 model here: https://gist.github.com/simonw/de4951452df2677f2a1a3cd415168b60 https://gist.github.com/simonw/de4951452df2677f2a1a3cd415168... I'm not going to try for an extensive evaluation comparing it with Llama 3.3 though, life's too short and that's already been done better than I could by https://livebench.ai/ https://livebench.ai/
- MichaelZuo 2y agoWhy not ask it to solve math questions? The bar for GPT-4 was so low that unambiguously clearing that threshold should be pretty easy.
- simonw 2y agoI am not particularly interested in those benchmarks that deliberately expose weaknesses in models: I know that models have weaknesses already! What I care about is the things that they're proven to be good at - can I do those kinds of things (RAG, summarization, code generation, language translation) directly on my laptop?