3 ms·
Well the first paper's authors does say: "While our ability to interpret these results is limited by nascent scientific understanding of LLMs and the proprietar
by MichaelZuo 4y ago
Well the first paper's authors does say: "While our ability to interpret these results is limited by nascent scientific understanding of LLMs and the proprietary nature of GPT, we believe that these results strongly suggest that an LLM will pass the MBE component of the Bar Exam in the near future."
- JohnFen 4y agoWhich is an overt admission that LLMs can't pass the bar exam right now.
- MichaelZuo 4y agoChatGPT 2.0 being able to do so would still be pretty relevant.
- dragonwriter 4y agoAnd if ChatGPT 2.0 does, than the future article that claims that stock ChatGPT 2.0 can will be justified. But the current article that claims that the stock ChatGPT of today can do that will still be wrong.
- qualudeheart 4y agoYou should see the capacities from Chinese LLMs.
- dragonwriter 4y agoYes, so the researchers, based on ChatGPT’s failure, predict that some other LLM in the new future will pass the same subset of the bar exam that ChatGPT failed to pass. Which is nice, but very much not support for the article’s claim that stock ChatGPT can, already, pass the bar exam. There is a pretty big gap between “passing the bar exam” and “giving researchers a feeling of optimism that some other system will pass a particular subset of the bar exam in the ’near future’”. ChatGPT has demonstrated the ability to do the latter, not the former.
- elcomet 4y agoIt's not based on ChatGPT,s failure, it's based on ChatGPT's progress that they predict this.
- qualudeheart 4y agoPerformance soon will be much better and many a legal professional will go back to grad school.