3 ms·
In my experience GPT labeling is good, but it makes errors, about 10% in my case. Maybe it's better on average than a human, but not perfect for the task. I was
by visarga 4y ago
In my experience GPT labeling is good, but it makes errors, about 10% in my case. Maybe it's better on average than a human, but not perfect for the task. I was doing open-ended schema matching, a hard task because of the vast number of filed names.
- kolinko 4y agoDid you try using multiple AIs for the same task (e.g. GPT & llama), or GPT with two-three prompts?
- visarga 4y agoYes, it contradicts itself if you call it multiple times. So I get to measure ensemble agreement as a proxy for correctness. But what do I do with the examples where the ensemble is not confident? They are too many to label by hand, and a pity to throw out - they could be the most interesting training examples.