3 ms·
We need public benchmarks. This is incredibly fast progress on large contexts and I would like to see if they are actually attending equally as well to all of
by whimsicalism 4y ago
We need public benchmarks.
This is incredibly fast progress on large contexts and I would like to see if they are actually attending equally as well to all of the information or there is some sparse approximation leading to intelligence/reasoning degradation.
- monlockandkey 4y agohttps://lmsys.org/blog/2023-05-10-leaderboard/ https://lmsys.org/blog/2023-05-10-leaderboard/ https://chat.lmsys.org/?arena https://chat.lmsys.org/?arena Claude by Anthropic has more favourable responses then ChatGPT
- polishdude20 4y agoSo I tried this prompt in their chatbot arena multiple times. Each time getting the wrong answer: "Given that Beth is Sue's sister and Arnold is Sue's father and Beth Junior is Beth's Daughter and Jacob is Arnold's Great Grandfather, who is Jacob to Beth Junior?"
- Workaccount2 4y agoChatGPT3.5* It's still below GPT4, but it is closer to 4 than 3.5