7 ms·

Cops are using AI chatbots to write crime reports. Will they hold up in court?

76 points by djoldman 2y ago

theamk 2y ago
This is scary. There will be inevitable hallucinations, but this time they might land innocent people in jail.
- sixothree 2y ago
  On top of that, there is the issue of training data. We know police are often less than truthful or complete in their reports. Will these tools follow suit.
- danielmarkbruce 2y ago
  You assume current reports are better. That's probably a bad assumption. So many current fears around AI are based on the fact they might produce an error, as though the status quo is error free. It's not in most fields.
- superfrank 2y ago
  I have an acquaintance who is in a position at a police force that got a very early preview of this. With me working in tech, he wanted my opinion on it and hallucinations were my first concern. I'm not sure if it's still the case, but at that time the way the feature was set up it would add a few sentences of incorrect information into the report intentionally and the report would not be able to be submitted if that information was still in the report. I have no idea if that's how it works now or how well that works, but it seems like the team building this is aware of that possibility and at least taking some steps to try and prevent that from happening.
  - rurp 2y ago
    > it would add a few sentences of incorrect information into the report intentionally and the report would not be able to be submitted if that information was still in the report. This is an interesting safety feature but it would be better if the report submission went through and then resulted in disciplinary measures for the officer who submitted it. Simply blocking the submission makes it too easy to keep submitting the report with small edits in between, while never carefully verifying the full report.
- echoangle 2y ago
  Hopefully the person responsible for writing it reads the generated report and signs that it is accurate.
- johnisgood 2y ago
  Working as intended.
more_corn 2y ago
Ai chatbots struggle conforming to facts. This will end badly.
- nerdjon 2y ago
  Hopefully they review it very carefully and not well after the event so the memory is fresh. Otherwise, you may forget important details and just think it was correct if you need to reference it later. We know how influential someone's memory can be. There is something to be said about needing to write this stuff down fully to commit it to memory. I agree that this is likely going to end badly.
- isubkhankulov 2y ago
  Using LLMs to write a first draft based on your notes, rephrase and reword existing text is completely different from using an LLM to recall facts in its training data.
  - russdpale 2y ago
    It's using it's training data to rephrase and reword your existing text. Hello, mcfly.
- LargeWu 2y ago
  You think AI chatbots are bad with facts, you should see the cops! Honestly I don't think this will be any worse than police filing hand-written reports rife with their own biases and material omissions and sometimes even outright, intentional lies.
  - vundercind 2y ago
    I’d almost support requiring reports to go through an LLM filter that rewrites cop-speak and the exonerative voice into normal, straightforward language.
    - visarga 2y ago
hypeatei 2y ago
I honestly don't see an issue with this. This is the type of work that AI is made for: augmenting human tasks. Someone still reviews this before final submission (because court) The article even mentions officers being more thoughtful with words during the stops so that it can be summarized easily later on.
- ribosometronome 2y ago
  The only concern that really resonated with me is: >“I am concerned that automation and the ease of the technology would cause police officers to be sort of less careful with their writing,” said Ferguson, a law professor at American University working on what’s expected to be the first law review article on the emerging technology. >Ferguson said a police report is important in determining whether an officer’s suspicion “justifies someone’s loss of liberty.” It’s sometimes the only testimony a judge sees, especially for misdemeanor crimes. With the worry being that an officer might not catch an error that the LLM hallucinated or that they'll just overly trust the AI and not give it earnest review. That said, this sort of summarization seems to be something LLMs do well, I suspect it'll be more accurate than humans in the long run if it isn't already.
- kmeisthax 2y ago
  "Human in the loop" is not a sufficient control for this task, for several reasons: 1. The majority of criminal cases do not actually go to trial in the US. The charges are bargained down in plea deals. 2. In cases where the case does go to court, if an AI has made a mistake comprehending the recordings, or bullshitted its way through writing a crime report, that puts the onus on the defendant to actually find the mistake. LLMs are masters at bullshitting, you need a critical eye to spot the mistakes. Furthermore, those mistakes don't just need to be found, they actually need to be fed back into fine-tuning, or they don't get fixed. And there's no way any of that happens in this scenario.
- bee_rider 2y ago
  I don’t really get the workflow or how the AI helps. Whatever the cop is feeding to the AI, if the AI can make a narrative out of it, then that thing is what should be in the record. A loop where the cop generates prose with the help of an AI is bad because * the AI might lead the cop * more superfluous prose (which is all the AI can possibly generate because it doesn’t have any actual information about the case other than what the cop feeds it) is bad, it just wastes people’s time. The point isn’t to create beautiful stories, it is to make a record of the cop’s observations. It might be the case that we’re making the cops generate a bunch of prose. If that’s the case: 1) maybe it is done for a reason (like to make them sit down and consider what they observed). Circumventing that by just having them jot down off the cuff stuff and having an AI do the “sit down and consider” part defeats the whole process. 2) maybe it is a stupid task and we should consider just asking them for a couple bullet points.
bko 2y ago
I think the right balance is one of liability. There is a person responsible and can be creative as to how he creates the report. But if there is something wrong with the report, he would be responsible same as if he wrote the report himself. I for one welcome new technology to reduce human error and biases. This I don't understand: > He said automating those reports will “ease the police’s ability to harass, surveil and inflict violence on community members. While making the cop’s job easier, it makes Black and brown people’s lives harder.” This premise is anything that makes a police officers job easier means they can do more policing, which equates to making the lives of some people worse?
- janice1999 2y ago
  > I for one welcome new technology to reduce human error and biases. That assumes the technology does not contain those very biases, which we know is exactly what happens with LLMs. Maybe Axon have fair and unbiased datasets but unless they are forced to reveal that in court, no one will ever know.
  - bko 2y ago
    I think its easier to program LLMs than humans. You can change the dataset, or have explicit prompts to remove biases. Just look at how Google got their LLMs to re-imagine the founding fathers to remove bias.
    - janice1999 2y ago
      > You can change the dataset To what though? Datasets are not easy to create - why do you think they are almost all based on archives of stolen books and videos? Is Axon really going to spend millions to curate billions of words of English conversations free of racial bias for example? The same goes for voice recognition. Is the model for a woman with a thick regional accent ever going to be as good as a man speaking British English without significant investment? > Just look at how Google got their LLMs to re-imagine the founding fathers to remove bias. I can't tell if this is sarcasm or not since it is such a perfect example of unintended consequences and hubris.
golergka 2y ago
LLMs are an excellent text processing tool. But it doesn't matter what tools you used, you're responsible for whatever you wrote.
- deleted 2y ago
  [deleted]
- moralestapia 2y ago
  This, I'm fine with it as long as whoever signs it becomes liable for the consequences. i.e. if the AI makes up some shit and someone brings this to a court house, then perjury should apply as usual.
  - sixothree 2y ago
    Police have pretty much zero liability for just about anything they do. And if history is any guide that includes liability through lack of prosecution for blatantly illegal things as well.
  - Suppafly 2y ago
    >then perjury should apply as usual Unfortunately, it basically never applies even in the most blatant cases. Judges and prosecutors aren't trying to jam up their police buddies.
- HeatrayEnjoyer 2y ago
  >cops >being held responsible for anything at all I sure wish.
russdpale 2y ago
This is an absolutely terrible idea, people's lives are going to be ruined by this.
riiii 2y ago
I've seen a policeman write a crime report. He cherry picked and refurbished facts in a way that AI would have blushed. Perhaps the AI transcription isn't such a bad idea if the original output is preserved as a diff to the policeman's report.
- Terr_ 2y ago
  The problem is that there are things which won't be in the camera-record which still need to be recorded ASAP, without the risk of being lost/tampered-with by a random hallucination algorithm... Heck, even a human assistant saying "Let me write that for you" would be a problem. For example, there may be things the officer (claims) they quietly observed outside the scope of the camera. There needs to be a record of any of those which may be significant. ("Oh, that happened, but blame the LLM for the fact that it's not in the record.") It also extends to "I was justified doing X because Y" logic from the brain of the officer, even if generated after-the-fact. Just imagine the LLM as Microsoft Clippy: "Hi! It looks like you're trying to retroactively justify an unreasonable search!"
  - euroderf 2y ago
    > "Hi! It looks like you're trying to retroactively justify an unreasonable search!" I wouldn't mind a Federal Court -level LLM that flags anything that looks like parallel construction.
- pempem 2y ago
  AI can't blush. Also we've seen cops literally arrest sober people for DUIs. Adding AI doesn't really make me more excited
  - tim333 2y ago
    The "a body camera captured every word" might help.
    - deleted 2y ago
      [deleted]
speckx 2y ago
Yes. Until it's challenged.
qingcharles 2y ago
I feel qualified to comment on this as I've read and dissected thousands of police reports. 1) Most police reports are never read after they are written. Literally by nobody. Not the cop, not their superior, not the prosecutor, not the defense, not the court. Most criminal cases end in plea deals and the police reports will never be pulled or looked at. Most charges are filed by an on-duty low-level prosecutor who fields phone calls all day from officers who re-iterate the facts of the crime and ask for the statutes that have been violated. 2) Police have to testify at a trial, usually from memory. This often happens one, two, three years or more after they wrote the report. They are usually handed their old report outside the courtroom before they go in to refresh their memory. This is problematic as they will naturally conform their sworn testimony to whatever is written on the report. 3) Police reports right now are VERY badly written. They are impossibly short, impossibly vague and ridiculously low on detail. If you go back to some code you wrote last year and wonder what the hell it does because you didn't comment any of it, it's a bit like that. 4) Police reports are usually horseshit, for various reasons. They tell one side of a story. The suspect will have another view. The truth generally lies somewhere in the middle. 5) I think this might help, because the AI is naturally more neutral than an officer who's whole job is to apprehend criminals and their natural tendency is to make themselves look good and just and right and the suspect look like a really horrible human being. 6) The problem is that the AI might mishear, misunderstand or just plain hallucinate. At first the officers will re-read it, but after a few of these they will get lazy and just click OK on every report. I've been using AI to generate Alt Text for images and I've been getting lazy at checking it. Just yesterday it told me this cup[0] said "COFFEE" on it, which is a good guess, but it actually says OCEAN FEELS. It guessed, and it was 100% sure it was right. Just my two centimes. [0] https://imgur.com/a/9UEtzAD https://imgur.com/a/9UEtzAD
- Terr_ 2y ago
  Another problem: "The police officer generated a vague half-assed report with no good explanation for what they did" can itself be evidence in a trial. So running that through an LLM--or even delegating it to a human assistant--is kind of like tampering. It can easily generates falsehoods for a jury, like "this officer has clear recall of events" or "the officer is diligent in fact-checking" or "the officer had a proper reason for what they did". Sure, an author could try to get the same result on their own, but at least it's the same brain and not ghostwritten by someone who wasn't there, and they can't just shift the blame for contradictions onto The Magic Mystery Machine.
JCM9 2y ago
The challenge here is the broader challenge with AI (eg self driving cars). Folks want the benefits but while also absolving themselves of the responsibility, liability and accountability for the quality of the output. That’s a fundamental issue with so many of the proposed use cases for these things. You can’t throw an AI in jail for filing a false police report. Yes you can simply say officer must review but we all know folks are going to cut corners. Until someone goes to jail for filing a false police report and fails with a “but the AI did it” defense, I’d expect full blown shenanigans here. We got a preview of this with the lawyer that filed an error ridden court filing and the judge threw the book at the author attorney and wasn’t having any of his “but the AI wrote it” defense.
- katbyte 2y ago
  Easy solution, people are responsible for signing off on what the ai spits out.
  - ceejayoz 2y ago
    We can't even manage that for the police reports cops write by hand. http://www.nytimes.com/2021/08/28/us/false-police-statements.html http://www.nytimes.com/2021/08/28/us/false-police-statements... / https://archive.is/1fCet https://archive.is/1fCet
    - exe34 2y ago
      they can even shoot unarmed you and then claim immunity. what's a bit of paperwork?
silveira 2y ago
Cops will be fine, I can tell you that.
jcims 2y ago
What might be better is have a large language model interview the officer and the whole thing recorded and transcribed.
- bee_rider 2y ago
  Seems possible the LLM could lead the officer in unnecessary directions. Could a machine interview be replaced by a checklist or flowchart, I wonder? These LLM’s that can perform actions that look like conversing and thinking always seem to prompt the question: why did we do it that way? IMO the point of an interview is often not to just get the questions answered, but to get a second set of eyeballs with responsibility, agency, and the ability to blow the whistle, to scrutinize the series of events. Unless an AI is created with those abilities, I don’t see the point in having it interview people.
  - jcims 2y ago
    Great point. All it would take is some evidence of bias in the follow up questions to invalidate that approach completely.
- c-linkage 2y ago
  Why stop there? Train ChatGPT on the legal code then feed live video into it. ChatGPT will then fine violators immediately for those crimes that are fineable. For arrestable crimes have it write the summons and then use text to speech so it can radio a police officer and dispatch to your location. The officer doesn't even have to know why they are arresting you, and it means they can be completely unaware of the legal code because the infraction was already detected and adjudicated.
  - jrflowers 2y ago
    The other day an LLM told me that the singer Charli XCX’s real name is “Professor Charles Xavier”
caseyy 2y ago
This is very multi-faceted and complex. From one perspective, there should be no harm in simply using LLMs as a text processor, if the accounts written down in said text are genuine and verified as such by the officer. From a more practical perspective, an account written by a witness itself will always be more accurate to what they actually saw and how they interpreted it, than a statement written by any third party. It doesn’t matter if the third party is human or not. Also, LLMs show heavy Silicon Valley ethical bias. This is not news to anyone here. To ingest this bias into our legal system, I think grants too much power to the tech companies. Especially in common law, where precedents can be established. On the other hand, the justice system will be largely supervised by humans, so perhaps they will discern what is right and wrong, moral and immoral, ethical and unethical, or what has perverted ethics. Then again, if this system becomes more AI-based over time, we may lose human control and our legal system may be in large part controlled by tech companies. This is not a good slope to slide down, and how slippery it is remains to be seen. Also, what if the halo effect created by the way LLM expresses itself hides institutional prejudice? You can always push ChatGPT to come up with 20 polite ways to express your racist beliefs, for example. But if you wrote your own statements, they may be more visible. I’m sure police already know whatever LLMs produce is more PR friendly as a benefit, but is there a dark side to this? Lastly, what about data protection? What if I don’t want my data to be ingested by LLMs for training, but I am, let’s say, a victim of a crime? Do I have no reasonable expectation of privacy anymore? Remember what Google said about people who hand over their data to third parties and how they argued it is evidence that such people don’t have a reasonable expectation of privacy. The responsible adult thing to do would be to run research on it and find how the LLMs bias police statements and what kind of effects that can have to a justice system — proper double blind studies. Done by a committee of experts from judges to institutes and projects for civil justice. With so many ways to slip off the thin tightrope where everyone acts with integrity and not only best interest, but also competent best interest of one another and justice… I think this is bound to end poorly if we go balls to the walls with this sort of thing. And then to top this off, other commenters present strong arguments that further complicate the matter.
Animats 2y ago
This does create evidence problems. The chatbot result should be alongside the video, with links to the appropriate section of the video. Then the defense can look for discrepancies and the judge can easily check them.
kazinator 2y ago
I suspect crime reports with spelling and grammar fixed by AI will might hold up better in court than unassisted reports.

76 points by djoldman 2y ago

theamk 2y ago

This is scary. There will be inevitable hallucinations, but this time they might land innocent people in jail.

sixothree 2y ago
On top of that, there is the issue of training data. We know police are often less than truthful or complete in their reports. Will these tools follow suit.
danielmarkbruce 2y ago
You assume current reports are better. That's probably a bad assumption. So many current fears around AI are based on the fact they might produce an error, as though the status quo is error free. It's not in most fields.
superfrank 2y ago
I have an acquaintance who is in a position at a police force that got a very early preview of this. With me working in tech, he wanted my opinion on it and hallucinations were my first concern. I'm not sure if it's still the case, but at that time the way the feature was set up it would add a few sentences of incorrect information into the report intentionally and the report would not be able to be submitted if that information was still in the report. I have no idea if that's how it works now or how well that works, but it seems like the team building this is aware of that possibility and at least taking some steps to try and prevent that from happening.
- rurp 2y ago
  > it would add a few sentences of incorrect information into the report intentionally and the report would not be able to be submitted if that information was still in the report. This is an interesting safety feature but it would be better if the report submission went through and then resulted in disciplinary measures for the officer who submitted it. Simply blocking the submission makes it too easy to keep submitting the report with small edits in between, while never carefully verifying the full report.
echoangle 2y ago
Hopefully the person responsible for writing it reads the generated report and signs that it is accurate.
johnisgood 2y ago
Working as intended.

more_corn 2y ago

Ai chatbots struggle conforming to facts. This will end badly.

nerdjon 2y ago
Hopefully they review it very carefully and not well after the event so the memory is fresh. Otherwise, you may forget important details and just think it was correct if you need to reference it later. We know how influential someone's memory can be. There is something to be said about needing to write this stuff down fully to commit it to memory. I agree that this is likely going to end badly.
isubkhankulov 2y ago
Using LLMs to write a first draft based on your notes, rephrase and reword existing text is completely different from using an LLM to recall facts in its training data.
- russdpale 2y ago
  It's using it's training data to rephrase and reword your existing text. Hello, mcfly.
LargeWu 2y ago
You think AI chatbots are bad with facts, you should see the cops! Honestly I don't think this will be any worse than police filing hand-written reports rife with their own biases and material omissions and sometimes even outright, intentional lies.
- vundercind 2y ago
  I’d almost support requiring reports to go through an LLM filter that rewrites cop-speak and the exonerative voice into normal, straightforward language.
  - visarga 2y ago

hypeatei 2y ago

I honestly don't see an issue with this. This is the type of work that AI is made for: augmenting human tasks. Someone still reviews this before final submission (because court) The article even mentions officers being more thoughtful with words during the stops so that it can be summarized easily later on.

ribosometronome 2y ago
The only concern that really resonated with me is: >“I am concerned that automation and the ease of the technology would cause police officers to be sort of less careful with their writing,” said Ferguson, a law professor at American University working on what’s expected to be the first law review article on the emerging technology. >Ferguson said a police report is important in determining whether an officer’s suspicion “justifies someone’s loss of liberty.” It’s sometimes the only testimony a judge sees, especially for misdemeanor crimes. With the worry being that an officer might not catch an error that the LLM hallucinated or that they'll just overly trust the AI and not give it earnest review. That said, this sort of summarization seems to be something LLMs do well, I suspect it'll be more accurate than humans in the long run if it isn't already.
kmeisthax 2y ago
"Human in the loop" is not a sufficient control for this task, for several reasons: 1. The majority of criminal cases do not actually go to trial in the US. The charges are bargained down in plea deals. 2. In cases where the case does go to court, if an AI has made a mistake comprehending the recordings, or bullshitted its way through writing a crime report, that puts the onus on the defendant to actually find the mistake. LLMs are masters at bullshitting, you need a critical eye to spot the mistakes. Furthermore, those mistakes don't just need to be found, they actually need to be fed back into fine-tuning, or they don't get fixed. And there's no way any of that happens in this scenario.
bee_rider 2y ago
I don’t really get the workflow or how the AI helps. Whatever the cop is feeding to the AI, if the AI can make a narrative out of it, then that thing is what should be in the record. A loop where the cop generates prose with the help of an AI is bad because * the AI might lead the cop * more superfluous prose (which is all the AI can possibly generate because it doesn’t have any actual information about the case other than what the cop feeds it) is bad, it just wastes people’s time. The point isn’t to create beautiful stories, it is to make a record of the cop’s observations. It might be the case that we’re making the cops generate a bunch of prose. If that’s the case: 1) maybe it is done for a reason (like to make them sit down and consider what they observed). Circumventing that by just having them jot down off the cuff stuff and having an AI do the “sit down and consider” part defeats the whole process. 2) maybe it is a stupid task and we should consider just asking them for a couple bullet points.

bko 2y ago

I think the right balance is one of liability. There is a person responsible and can be creative as to how he creates the report. But if there is something wrong with the report, he would be responsible same as if he wrote the report himself. I for one welcome new technology to reduce human error and biases. This I don't understand: > He said automating those reports will “ease the police’s ability to harass, surveil and inflict violence on community members. While making the cop’s job easier, it makes Black and brown people’s lives harder.” This premise is anything that makes a police officers job easier means they can do more policing, which equates to making the lives of some people worse?

janice1999 2y ago
> I for one welcome new technology to reduce human error and biases. That assumes the technology does not contain those very biases, which we know is exactly what happens with LLMs. Maybe Axon have fair and unbiased datasets but unless they are forced to reveal that in court, no one will ever know.
- bko 2y ago
  I think its easier to program LLMs than humans. You can change the dataset, or have explicit prompts to remove biases. Just look at how Google got their LLMs to re-imagine the founding fathers to remove bias.
  - janice1999 2y ago
    > You can change the dataset To what though? Datasets are not easy to create - why do you think they are almost all based on archives of stolen books and videos? Is Axon really going to spend millions to curate billions of words of English conversations free of racial bias for example? The same goes for voice recognition. Is the model for a woman with a thick regional accent ever going to be as good as a man speaking British English without significant investment? > Just look at how Google got their LLMs to re-imagine the founding fathers to remove bias. I can't tell if this is sarcasm or not since it is such a perfect example of unintended consequences and hubris.

golergka 2y ago

LLMs are an excellent text processing tool. But it doesn't matter what tools you used, you're responsible for whatever you wrote.

deleted 2y ago
[deleted]
moralestapia 2y ago
This, I'm fine with it as long as whoever signs it becomes liable for the consequences. i.e. if the AI makes up some shit and someone brings this to a court house, then perjury should apply as usual.
- sixothree 2y ago
  Police have pretty much zero liability for just about anything they do. And if history is any guide that includes liability through lack of prosecution for blatantly illegal things as well.
- Suppafly 2y ago
  >then perjury should apply as usual Unfortunately, it basically never applies even in the most blatant cases. Judges and prosecutors aren't trying to jam up their police buddies.
HeatrayEnjoyer 2y ago
>cops >being held responsible for anything at all I sure wish.

russdpale 2y ago

This is an absolutely terrible idea, people's lives are going to be ruined by this.

riiii 2y ago

I've seen a policeman write a crime report. He cherry picked and refurbished facts in a way that AI would have blushed. Perhaps the AI transcription isn't such a bad idea if the original output is preserved as a diff to the policeman's report.

Terr_ 2y ago
The problem is that there are things which won't be in the camera-record which still need to be recorded ASAP, without the risk of being lost/tampered-with by a random hallucination algorithm... Heck, even a human assistant saying "Let me write that for you" would be a problem. For example, there may be things the officer (claims) they quietly observed outside the scope of the camera. There needs to be a record of any of those which may be significant. ("Oh, that happened, but blame the LLM for the fact that it's not in the record.") It also extends to "I was justified doing X because Y" logic from the brain of the officer, even if generated after-the-fact. Just imagine the LLM as Microsoft Clippy: "Hi! It looks like you're trying to retroactively justify an unreasonable search!"
- euroderf 2y ago
  > "Hi! It looks like you're trying to retroactively justify an unreasonable search!" I wouldn't mind a Federal Court -level LLM that flags anything that looks like parallel construction.
pempem 2y ago
AI can't blush. Also we've seen cops literally arrest sober people for DUIs. Adding AI doesn't really make me more excited
- tim333 2y ago
  The "a body camera captured every word" might help.
  - deleted 2y ago
    [deleted]

speckx 2y ago

Yes. Until it's challenged.

qingcharles 2y ago

I feel qualified to comment on this as I've read and dissected thousands of police reports. 1) Most police reports are never read after they are written. Literally by nobody. Not the cop, not their superior, not the prosecutor, not the defense, not the court. Most criminal cases end in plea deals and the police reports will never be pulled or looked at. Most charges are filed by an on-duty low-level prosecutor who fields phone calls all day from officers who re-iterate the facts of the crime and ask for the statutes that have been violated. 2) Police have to testify at a trial, usually from memory. This often happens one, two, three years or more after they wrote the report. They are usually handed their old report outside the courtroom before they go in to refresh their memory. This is problematic as they will naturally conform their sworn testimony to whatever is written on the report. 3) Police reports right now are VERY badly written. They are impossibly short, impossibly vague and ridiculously low on detail. If you go back to some code you wrote last year and wonder what the hell it does because you didn't comment any of it, it's a bit like that. 4) Police reports are usually horseshit, for various reasons. They tell one side of a story. The suspect will have another view. The truth generally lies somewhere in the middle. 5) I think this might help, because the AI is naturally more neutral than an officer who's whole job is to apprehend criminals and their natural tendency is to make themselves look good and just and right and the suspect look like a really horrible human being. 6) The problem is that the AI might mishear, misunderstand or just plain hallucinate. At first the officers will re-read it, but after a few of these they will get lazy and just click OK on every report. I've been using AI to generate Alt Text for images and I've been getting lazy at checking it. Just yesterday it told me this cup[0] said "COFFEE" on it, which is a good guess, but it actually says OCEAN FEELS. It guessed, and it was 100% sure it was right. Just my two centimes. [0] https://imgur.com/a/9UEtzAD https://imgur.com/a/9UEtzAD

Terr_ 2y ago
Another problem: "The police officer generated a vague half-assed report with no good explanation for what they did" can itself be evidence in a trial. So running that through an LLM--or even delegating it to a human assistant--is kind of like tampering. It can easily generates falsehoods for a jury, like "this officer has clear recall of events" or "the officer is diligent in fact-checking" or "the officer had a proper reason for what they did". Sure, an author could try to get the same result on their own, but at least it's the same brain and not ghostwritten by someone who wasn't there, and they can't just shift the blame for contradictions onto The Magic Mystery Machine.

JCM9 2y ago

The challenge here is the broader challenge with AI (eg self driving cars). Folks want the benefits but while also absolving themselves of the responsibility, liability and accountability for the quality of the output. That’s a fundamental issue with so many of the proposed use cases for these things. You can’t throw an AI in jail for filing a false police report. Yes you can simply say officer must review but we all know folks are going to cut corners. Until someone goes to jail for filing a false police report and fails with a “but the AI did it” defense, I’d expect full blown shenanigans here. We got a preview of this with the lawyer that filed an error ridden court filing and the judge threw the book at the author attorney and wasn’t having any of his “but the AI wrote it” defense.

katbyte 2y ago
Easy solution, people are responsible for signing off on what the ai spits out.
- ceejayoz 2y ago
  We can't even manage that for the police reports cops write by hand. http://www.nytimes.com/2021/08/28/us/false-police-statements.html http://www.nytimes.com/2021/08/28/us/false-police-statements... / https://archive.is/1fCet https://archive.is/1fCet
  - exe34 2y ago
    they can even shoot unarmed you and then claim immunity. what's a bit of paperwork?

silveira 2y ago

Cops will be fine, I can tell you that.

jcims 2y ago

What might be better is have a large language model interview the officer and the whole thing recorded and transcribed.

bee_rider 2y ago
Seems possible the LLM could lead the officer in unnecessary directions. Could a machine interview be replaced by a checklist or flowchart, I wonder? These LLM’s that can perform actions that look like conversing and thinking always seem to prompt the question: why did we do it that way? IMO the point of an interview is often not to just get the questions answered, but to get a second set of eyeballs with responsibility, agency, and the ability to blow the whistle, to scrutinize the series of events. Unless an AI is created with those abilities, I don’t see the point in having it interview people.
- jcims 2y ago
  Great point. All it would take is some evidence of bias in the follow up questions to invalidate that approach completely.
c-linkage 2y ago
Why stop there? Train ChatGPT on the legal code then feed live video into it. ChatGPT will then fine violators immediately for those crimes that are fineable. For arrestable crimes have it write the summons and then use text to speech so it can radio a police officer and dispatch to your location. The officer doesn't even have to know why they are arresting you, and it means they can be completely unaware of the legal code because the infraction was already detected and adjudicated.
- jrflowers 2y ago
  The other day an LLM told me that the singer Charli XCX’s real name is “Professor Charles Xavier”

caseyy 2y ago

This is very multi-faceted and complex. From one perspective, there should be no harm in simply using LLMs as a text processor, if the accounts written down in said text are genuine and verified as such by the officer. From a more practical perspective, an account written by a witness itself will always be more accurate to what they actually saw and how they interpreted it, than a statement written by any third party. It doesn’t matter if the third party is human or not. Also, LLMs show heavy Silicon Valley ethical bias. This is not news to anyone here. To ingest this bias into our legal system, I think grants too much power to the tech companies. Especially in common law, where precedents can be established. On the other hand, the justice system will be largely supervised by humans, so perhaps they will discern what is right and wrong, moral and immoral, ethical and unethical, or what has perverted ethics. Then again, if this system becomes more AI-based over time, we may lose human control and our legal system may be in large part controlled by tech companies. This is not a good slope to slide down, and how slippery it is remains to be seen. Also, what if the halo effect created by the way LLM expresses itself hides institutional prejudice? You can always push ChatGPT to come up with 20 polite ways to express your racist beliefs, for example. But if you wrote your own statements, they may be more visible. I’m sure police already know whatever LLMs produce is more PR friendly as a benefit, but is there a dark side to this? Lastly, what about data protection? What if I don’t want my data to be ingested by LLMs for training, but I am, let’s say, a victim of a crime? Do I have no reasonable expectation of privacy anymore? Remember what Google said about people who hand over their data to third parties and how they argued it is evidence that such people don’t have a reasonable expectation of privacy. The responsible adult thing to do would be to run research on it and find how the LLMs bias police statements and what kind of effects that can have to a justice system — proper double blind studies. Done by a committee of experts from judges to institutes and projects for civil justice. With so many ways to slip off the thin tightrope where everyone acts with integrity and not only best interest, but also competent best interest of one another and justice… I think this is bound to end poorly if we go balls to the walls with this sort of thing. And then to top this off, other commenters present strong arguments that further complicate the matter.

Animats 2y ago

This does create evidence problems. The chatbot result should be alongside the video, with links to the appropriate section of the video. Then the defense can look for discrepancies and the judge can easily check them.

kazinator 2y ago

I suspect crime reports with spelling and grammar fixed by AI will might hold up better in court than unassisted reports.