"GPT-4 passes the bar exam with a top 10% score." Turns out that may be as much marketing as truth. According to this paper[1]: "although GPT-4's UBE score n

"GPT-4 passes the bar exam with a top 10% score." Turns out that may be as much marketing as truth. According to this paper[1]: "although GPT-4's UBE score nears the 90th percentile when examining approximate conversions from February administrations of the Illinois Bar Exam, these estimates are heavily skewed towards repeat test-takers who failed the July administration and score significantly lower than the general test-taking population. Second, data from a recent July administration of the same exam suggests GPT-4's overall UBE percentile was ~68th percentile, and ~48th percentile on essays. Third, examining official NCBE data and using several conservative statistical assumptions, GPT-4's performance against first-time test takers is estimated to be ~63rd percentile, including ~41st percentile on essays. Fourth, when examining only those who passed the exam (i.e. licensed or license-pending attorneys), GPT-4's performance is estimated to drop to ~48th percentile overall, and ~15th percentile on essays." [1] - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4441311 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4441311