I've been looking at using llms to extract as structured information from structured but human written documents (local municipal codes). Unfortunately the mode

I've been looking at using llms to extract as structured information from structured but human written documents (local municipal codes). Unfortunately the models need a lot of handholding to do this without hallucinating or making generic statements. They produce output that looks correct and that's an accomplishment on its own. Unfortunately the output often has little bearing on the reality presented to it. Realistically... They need to evolve much more. As an experiment I paid people on fiver to do the exact same thing with the same prompt and they get it right. These are not Americans so the norms of American municipalities are new to them.