3 ms·
1000x this. Entity extraction from unstructured text with zero/few-shot is fantastic. I've got a use case where I need to extract model numbers from text - th
by celestialcheese 4y ago
1000x this. Entity extraction from unstructured text with zero/few-shot is fantastic.
I've got a use case where I need to extract model numbers from text - these LLMs are so good at it with very little work.
- Oras 4y agoBe careful with NER, while it is very good, it is not perfect. Example, I tried to extract skills from a job posting. ChatGPT did well, but there were skills missing. It is good to find some entities but then you need to extend the labeling manually.
- celestialcheese 4y agoFor sure. If perfect accuracy is important, it's still good to do sampling and human review to figure out accuracy rate and decide if further checking is required. But it still beats the pants off accuracy of other methods for the amount of work required. That said, with fine-tuning, `davinci-003` is _excellent_ at the types of entity extraction you're describing.
- dandiep 4y agoThere is no such thing as davinci-003 per se. There is text-davinci-003 which you can't fine tune, and davinci which you can.
- rolisz 4y agoSure, but SOTA on NER is around 90% for things like Names and Places. For skills, a Spacy model will get around 60-70%. And training a Spacy model takes a bit of fiddling.
- Oras 4y agoCheckout Argilla for annotation. You can use vectors to speed up the annotations and you can also start with zero-shot feedback to improve the training as you go https://www.argilla.io/ https://www.argilla.io/
- shostack 4y agoThe cost may be high but still worth it depending on the alternative. I wonder what is happening with this in the evidence mining tools lawyers use for example.