3 ms·
This is highly misleading. ChatGPT did not ace the bar exam -- it was basically percentile graded against a group of people who mostly failed. If compared to r
by habinero 2y ago
This is highly misleading.
ChatGPT did not ace the bar exam -- it was basically percentile graded against a group of people who mostly failed. If compared to real lawyers, it was 15th percentile on the essay portion
[0] https://law-ai.org/re-evaluating-gpt-4s-bar-exam-performance/ https://law-ai.org/re-evaluating-gpt-4s-bar-exam-performance...
- ben_w 2y agoI said pass, not ace. 15th percentile of passes, on the weakest aspect, is still a big improvement over "not passing". That improvement is what I wish to highlight. (The observation that 48th percentile (lowest overall from your link, let alone 15th for essays) of passes corresponds to 90th percentile of all exam takers, suggests that perhaps too many humans are taking the exams before they're ready).