If the answer is that each outcome of such model should be double checked by a human while going through all that paperwork then what's the point of having that

If the answer is that each outcome of such model should be double checked by a human while going through all that paperwork then what's the point of having that automation in the first place? That is the underlying issue here. The liability for an erroneous AI output is the same as errors committed by any other agent of the company: the company is on the hook, though it may have recourse against any third party who trained or operates the model. For domains such as law, where the penalty for hallucinations are extremely severe, and, conversely, the penalty for missing things is likely to be losing litigation, this means treating AI like any other LPO: a duly qualified attorney takes responsibility for the work, which means a bunch of associates doing varying levels of review. Is there still a place for AI in all this? Absolutely. But as long as hallucinations are a basic feature of the architecture, you’re going to spend as much human time on anything of critical importance as you otherwise would, and a lot of deterministic business rules to constrain the results of everything else.