I've been implementing batch processing for information extraction and schema matching with GPT3.5 and it works decently, but still has about 10% error rate, es

I've been implementing batch processing for information extraction and schema matching with GPT3.5 and it works decently, but still has about 10% error rate, especially when extracting many fields from a document. It is also expensive to use on millions of documents. The solution I see is distilling in smaller models. The problem I am facing now is how to filter out the mislabeled training data generated with GPT3, also around 10%. Manual validation is out of the question at this scale, and it seems to be the most interesting training data if I could have it validated. I tried GPT3 fixing itself, only works partially. What I did was to train a model and use that model to rank my data, and set a threshold. That's how I got the cutoff around 10%. I can use the easy part, but what to do with the interesting boundary region data that might be mislabeled?