4 ms·
I do wish they'd used some more objective criteria. Simply being preferable one of the things LLMs have trained for since the beginning, hence its sycophantic n
by jshier 4mo ago
I do wish they'd used some more objective criteria. Simply being preferable one of the things LLMs have trained for since the beginning, hence its sycophantic nature.
- wilg 4mo agoWhat criteria would you use for judging legal arguments?
- mylifeandtimes 4mo agomaybe seeing if the case law it cited was real or imagined? Just one idea, IANAL
- gamerDude 4mo agoWell, they had the data around if the answer would be harmful to the students learning. AI was scored at 3.5% harmful answers and 12% of law professor answers were considered harmful.
- mitkebes 4mo agoThe arguments need to be based on actual law, and any cited reference cases need to be real. There's been a lot of news stories about lawyers using AI, and then getting in trouble for citing hallucinated laws or cases. It doesn't matter if the AI response is "preferred" over the human one if it gets thrown out when put under the scrutiny of a real case.
- adornKey 4mo agoMaybe sycophantic nature is a good fit for the legal system. A successful lawyer once told me that the most important thing is to know your judge. Objectivity isn't a big thing in court. They'll cite random newspaper articles as evidence and throw out expert opinions - if they like. There might be a way to appeal - but that road often is not functional.