more datapoints ive been collecting for the latent.space recap: - lmsys ranks claude 3 opus ahead of GPT4 but behind GPT4T https://x.com/lmsysorg/status/17657

more datapoints ive been collecting for the latent.space recap: - lmsys ranks claude 3 opus ahead of GPT4 but behind GPT4T https://x.com/lmsysorg/status/1765774296000172289 https://x.com/lmsysorg/status/1765774296000172289 mistral large is also well behind claude 3 sonnet, and both behind gpt4, which is... good but disappointing?. - ai2 wildbench largely agrees with this ranking https://x.com/billyuchenlin/status/1766079601154064688?s=20 https://x.com/billyuchenlin/status/1766079601154064688?s=20 - claude summarization is better, at the cost of higher hallucination https://twitter.com/swyx/status/1764805626037993853 https://twitter.com/swyx/status/1764805626037993853 it's also noteworthy that gemini and claude seem to have completely solved the needle in a haystack problem vs openai/mistral - i'm not sure what breakthrough is responsible but the opensource community seems to be pointing at ringattention as the main promise