4 ms·

I write public policy and law, but have not found a competent AI that can replace what I can do. I think it is not so much about the "black letter of the law" b

I write public policy and law, but have not found a competent AI that can replace what I can do. I think it is not so much about the "black letter of the law" but in the way that humans may perceive and interpret it. That and the AI just gets a lot of things wrong, and can't craft a legal statute. Maybe someday but I think in law and policy it's best to avoid AI.

basch 2y ago
Chatbots with temperature are probably the wrong tool. An LLM that can translate english into formal proof, a logic engine that can apply law to events and situations, and then a translator that can turn the result back into English should be achievable. It all comes back to "trust but verify." Until it outperforms humans with less mistakes, it needs an editor reviewing the work closely. The main problem with current LLM output and humans right now is that the LLM outputs such verbose text, humans are overwhelmed and just go with it, instead of applying the time it saved them to reviewing the work, they just move on.
- add-sub-mul-div 2y ago
  "Trust but verify" isn't applicable in a situation where any sentence in a passage could be randomly false. If you have to verify every part of something, you'd have been better off doing the work yourself. (Unrelated, you're also better off doing work yourself because that's how you build expertise, but no one's really talking about that.)
  - basch 2y ago
    As a starting point to "return all relevant case law" its not all that different from search, except maybe better tailored or summarized.
- TazeTSchnitzel 2y ago
  Law cannot be translated into formal proofs, because it is designed to interface with the messyness of the real world, not a pure mathematical system, and always requires case-by-case contextual interpetations.
  - basch 2y ago
    "If this happens, then this happens." If the person running the software checks of that the event happened, the resulting consequence can be expected.
  - riku_iki 2y ago
    You can totally define various messyness math/logical formalizations, e.g. discretion, confidence score, etc.
  - Paul-Craft 2y ago
    Yes, and as I recently learned, there's an awful lot of law that only exists inside the heads of district court judges and law clerks. An argument that might win the day in one courtroom might fall flat in the next county over.
    - treetalker 2y ago
      In my experience, it's not that silent law exists only in judges' heads, so much as that the judges decide where they want to end up and then reason backward to create a decisional path that leads to the desired conclusion. This is, of course, blatantly wrong and not how the judicial system ought to work. For this reason (among others) we have the appellate process. Problems compound when the appellate panel does the same thing. Based on my experiences over years, both working in chambers and litigating, judges on a particular end of the spectrum tend to do this more than those on the other end.

basch 2y ago

Chatbots with temperature are probably the wrong tool. An LLM that can translate english into formal proof, a logic engine that can apply law to events and situations, and then a translator that can turn the result back into English should be achievable. It all comes back to "trust but verify." Until it outperforms humans with less mistakes, it needs an editor reviewing the work closely. The main problem with current LLM output and humans right now is that the LLM outputs such verbose text, humans are overwhelmed and just go with it, instead of applying the time it saved them to reviewing the work, they just move on.

add-sub-mul-div 2y ago
"Trust but verify" isn't applicable in a situation where any sentence in a passage could be randomly false. If you have to verify every part of something, you'd have been better off doing the work yourself. (Unrelated, you're also better off doing work yourself because that's how you build expertise, but no one's really talking about that.)
- basch 2y ago
  As a starting point to "return all relevant case law" its not all that different from search, except maybe better tailored or summarized.
TazeTSchnitzel 2y ago
Law cannot be translated into formal proofs, because it is designed to interface with the messyness of the real world, not a pure mathematical system, and always requires case-by-case contextual interpetations.
- basch 2y ago
  "If this happens, then this happens." If the person running the software checks of that the event happened, the resulting consequence can be expected.
- riku_iki 2y ago
  You can totally define various messyness math/logical formalizations, e.g. discretion, confidence score, etc.
- Paul-Craft 2y ago
  Yes, and as I recently learned, there's an awful lot of law that only exists inside the heads of district court judges and law clerks. An argument that might win the day in one courtroom might fall flat in the next county over.
  - treetalker 2y ago
    In my experience, it's not that silent law exists only in judges' heads, so much as that the judges decide where they want to end up and then reason backward to create a decisional path that leads to the desired conclusion. This is, of course, blatantly wrong and not how the judicial system ought to work. For this reason (among others) we have the appellate process. Problems compound when the appellate panel does the same thing. Based on my experiences over years, both working in chambers and litigating, judges on a particular end of the spectrum tend to do this more than those on the other end.