4 ms·
The discovery process for that case would be incredibly costly as it’s highly likely the AI would have been updated in between the incident and the court case.
by koolba 3y ago
The discovery process for that case would be incredibly costly as it’s highly likely the AI would have been updated in between the incident and the court case.
I bet if it were a private citizen running that AI, the overzealous DAs in NY would go after them for “destroying evidence of a crime” for not preserving the state of the AI.
- mmcwilliams 3y agoIn this case, where a RAG system is pulling from city legal code, what is the fix? Handling edge cases for individual laws it gets wrong?
- koolba 3y agoThe fix is not to rely on probabilistic text generation for legal purposes. It needs a verification step. Or maybe just use it to generate a set of canned answers for a FAQ, but again with each answer manually verified.
- mmcwilliams 3y agoI agree with you, I'm just unclear about the process by which people think this system could be updated for accuracy. I think manually verifying every possible answer means that the solution is to not use transformer models for this task.
- mcculley 3y agoI would expect most systems to preserve questions and answers for quality control and testing. Even if a system claims not to keep that record, I don’t trust it not to.