3 ms·
This will probably work as long as the material being annotated is similar to the material the LLM was trained on. When it encounters novel data (value) it will
by orangepurple 3y ago
This will probably work as long as the material being annotated is similar to the material the LLM was trained on. When it encounters novel data (value) it will likely perform poorly.
- nihit-desai 3y agoPartially agree, but it's a continuous value rather than a boolean. We've seen LLM performance largely follow this story: https://twitter.com/karpathy/status/1655994367033884672/photo/1 https://twitter.com/karpathy/status/1655994367033884672/phot... From benchmarking, we've been positively surprised by how effective few-shot learning and PEFT are, at closing the domain gap. "When it encounters novel data (value) it will likely perform poorly" -- is that not true of human annotators too? :)
- orangepurple 3y ago> "When it encounters novel data (value) it will likely perform poorly" -- is that not true of human annotators too? :) Some humans have intelligence and reasoning abilities. No LLMs do :)
- yawnxyz 3y agoI always regret making that assumption