3 ms·

I honestly don't see an issue with this. This is the type of work that AI is made for: augmenting human tasks. Someone still reviews this before final submissio

I honestly don't see an issue with this. This is the type of work that AI is made for: augmenting human tasks. Someone still reviews this before final submission (because court) The article even mentions officers being more thoughtful with words during the stops so that it can be summarized easily later on.

ribosometronome 2y ago
The only concern that really resonated with me is: >“I am concerned that automation and the ease of the technology would cause police officers to be sort of less careful with their writing,” said Ferguson, a law professor at American University working on what’s expected to be the first law review article on the emerging technology. >Ferguson said a police report is important in determining whether an officer’s suspicion “justifies someone’s loss of liberty.” It’s sometimes the only testimony a judge sees, especially for misdemeanor crimes. With the worry being that an officer might not catch an error that the LLM hallucinated or that they'll just overly trust the AI and not give it earnest review. That said, this sort of summarization seems to be something LLMs do well, I suspect it'll be more accurate than humans in the long run if it isn't already.
kmeisthax 2y ago
"Human in the loop" is not a sufficient control for this task, for several reasons: 1. The majority of criminal cases do not actually go to trial in the US. The charges are bargained down in plea deals. 2. In cases where the case does go to court, if an AI has made a mistake comprehending the recordings, or bullshitted its way through writing a crime report, that puts the onus on the defendant to actually find the mistake. LLMs are masters at bullshitting, you need a critical eye to spot the mistakes. Furthermore, those mistakes don't just need to be found, they actually need to be fed back into fine-tuning, or they don't get fixed. And there's no way any of that happens in this scenario.
bee_rider 2y ago
I don’t really get the workflow or how the AI helps. Whatever the cop is feeding to the AI, if the AI can make a narrative out of it, then that thing is what should be in the record. A loop where the cop generates prose with the help of an AI is bad because * the AI might lead the cop * more superfluous prose (which is all the AI can possibly generate because it doesn’t have any actual information about the case other than what the cop feeds it) is bad, it just wastes people’s time. The point isn’t to create beautiful stories, it is to make a record of the cop’s observations. It might be the case that we’re making the cops generate a bunch of prose. If that’s the case: 1) maybe it is done for a reason (like to make them sit down and consider what they observed). Circumventing that by just having them jot down off the cuff stuff and having an AI do the “sit down and consider” part defeats the whole process. 2) maybe it is a stupid task and we should consider just asking them for a couple bullet points.
remram 2y ago
Why does it need augmenting? If a bland list of facts is your report, this should be what is admitted in evidence. Reports don't need to be fancy.

ribosometronome 2y ago

The only concern that really resonated with me is: >“I am concerned that automation and the ease of the technology would cause police officers to be sort of less careful with their writing,” said Ferguson, a law professor at American University working on what’s expected to be the first law review article on the emerging technology. >Ferguson said a police report is important in determining whether an officer’s suspicion “justifies someone’s loss of liberty.” It’s sometimes the only testimony a judge sees, especially for misdemeanor crimes. With the worry being that an officer might not catch an error that the LLM hallucinated or that they'll just overly trust the AI and not give it earnest review. That said, this sort of summarization seems to be something LLMs do well, I suspect it'll be more accurate than humans in the long run if it isn't already.

kmeisthax 2y ago

"Human in the loop" is not a sufficient control for this task, for several reasons: 1. The majority of criminal cases do not actually go to trial in the US. The charges are bargained down in plea deals. 2. In cases where the case does go to court, if an AI has made a mistake comprehending the recordings, or bullshitted its way through writing a crime report, that puts the onus on the defendant to actually find the mistake. LLMs are masters at bullshitting, you need a critical eye to spot the mistakes. Furthermore, those mistakes don't just need to be found, they actually need to be fed back into fine-tuning, or they don't get fixed. And there's no way any of that happens in this scenario.

bee_rider 2y ago

I don’t really get the workflow or how the AI helps. Whatever the cop is feeding to the AI, if the AI can make a narrative out of it, then that thing is what should be in the record. A loop where the cop generates prose with the help of an AI is bad because * the AI might lead the cop * more superfluous prose (which is all the AI can possibly generate because it doesn’t have any actual information about the case other than what the cop feeds it) is bad, it just wastes people’s time. The point isn’t to create beautiful stories, it is to make a record of the cop’s observations. It might be the case that we’re making the cops generate a bunch of prose. If that’s the case: 1) maybe it is done for a reason (like to make them sit down and consider what they observed). Circumventing that by just having them jot down off the cuff stuff and having an AI do the “sit down and consider” part defeats the whole process. 2) maybe it is a stupid task and we should consider just asking them for a couple bullet points.

remram 2y ago

Why does it need augmenting? If a bland list of facts is your report, this should be what is admitted in evidence. Reports don't need to be fancy.