4 ms·

A friend of mine just defended his law PhD and in the introductory lectio said that (even) current LLMs would likely give better verdicts than human judges. Law

A friend of mine just defended his law PhD and in the introductory lectio said that (even) current LLMs would likely give better verdicts than human judges. Law isn't really a cognitively such demanding task as walking a dog or waiting tables.

numba888 2y ago
> current LLMs would likely give better verdicts He probably meant _brainwashed_ LLMs. They can consistently produce desired results if you wash them the right way. It's more about personal opinion than computation. Actually it would be fun to manipulate verdicts with prompt injections ;)
- jampekka 2y ago
  Judges are very much "brainwashed" too, and by design. The judges should apply the law, and the same case should ideally lead to the same verdict regardless of the judge. With the caveat that this applies to sane legal systems, and not the ones where "making examples" etc are part of the system.
  - numba888 2y ago
    > The judges should apply the law, and the same case should ideally lead to the same verdict regardless of the judge. hmm.. :) I like this. But the reality is very different and some factors which shouldn't matter can change the outcome dramatically. Like skin colors of defendant and judge. Pointing this out can be punished as well.
dgroshev 2y ago
This is nonsense though. What does "better" mean in this case? A judge is not a black box with an input (the case) and an output (the verdict), the entire point of having a judge is to have empathy, conscience, and personal responsibility built into the system. It's a blind spot that too many people have because we take those qualities for granted. LLMs unbundle them, so we need to start recognising the inherent value of humans, fast. I wrote a few words about it here: https://dgroshev.com/blog/feel-bad/ https://dgroshev.com/blog/feel-bad/
- jampekka 2y ago
  In civil law countries verdicts should not depend on an individual judge's feefees.
  - nyrikki 2y ago
    They should depend on individual case specifics, which is exactly what PAC learning is bad at.
  - dgroshev 2y ago
    A judgement is by necessity subjective. Have a look at some sentencing remarks, for example this, point 19 forward: https://www.judiciary.uk/wp-content/uploads/2021/09/Wayne-Couzens-Sentencing-Remarks.pdf https://www.judiciary.uk/wp-content/uploads/2021/09/Wayne-Co... Someone has to make a call. The weight of the call rests on the person's life experience, their understanding of the context and the cost to the society, their empathy to both the defendant and the accused, and their conscience. Treating it as a black box exercise misses the point completely.

numba888 2y ago

> current LLMs would likely give better verdicts He probably meant _brainwashed_ LLMs. They can consistently produce desired results if you wash them the right way. It's more about personal opinion than computation. Actually it would be fun to manipulate verdicts with prompt injections ;)

jampekka 2y ago
Judges are very much "brainwashed" too, and by design. The judges should apply the law, and the same case should ideally lead to the same verdict regardless of the judge. With the caveat that this applies to sane legal systems, and not the ones where "making examples" etc are part of the system.
- numba888 2y ago
  > The judges should apply the law, and the same case should ideally lead to the same verdict regardless of the judge. hmm.. :) I like this. But the reality is very different and some factors which shouldn't matter can change the outcome dramatically. Like skin colors of defendant and judge. Pointing this out can be punished as well.

dgroshev 2y ago

This is nonsense though. What does "better" mean in this case? A judge is not a black box with an input (the case) and an output (the verdict), the entire point of having a judge is to have empathy, conscience, and personal responsibility built into the system. It's a blind spot that too many people have because we take those qualities for granted. LLMs unbundle them, so we need to start recognising the inherent value of humans, fast. I wrote a few words about it here: https://dgroshev.com/blog/feel-bad/ https://dgroshev.com/blog/feel-bad/

jampekka 2y ago
In civil law countries verdicts should not depend on an individual judge's feefees.
- nyrikki 2y ago
  They should depend on individual case specifics, which is exactly what PAC learning is bad at.
- dgroshev 2y ago
  A judgement is by necessity subjective. Have a look at some sentencing remarks, for example this, point 19 forward: https://www.judiciary.uk/wp-content/uploads/2021/09/Wayne-Couzens-Sentencing-Remarks.pdf https://www.judiciary.uk/wp-content/uploads/2021/09/Wayne-Co... Someone has to make a call. The weight of the call rests on the person's life experience, their understanding of the context and the cost to the society, their empathy to both the defendant and the accused, and their conscience. Treating it as a black box exercise misses the point completely.