42 ms·
Data extraction is definitely one of the most useful functions of LLM, however, in my experience a large model is necessary for a reliable extraction - I tested
by druskacik 2y ago
Data extraction is definitely one of the most useful functions of LLM, however, in my experience a large model is necessary for a reliable extraction - I tested smaller, open-weights models and the performance was not sufficient.
I wonder, did anyone try to fine-tune a model specifically for general formatted data extraction? My naive thinking is that this should be pretty doable - after all, it's basically just restructuring the content using mostly the same tokens as input.
The reason why this would be useful (in my case) is because while large LLMs are perfectly capable of extraction, I often need to run it on millions of texts, which would be too costly. That's the reason I usually end up creating a custom small model, which is faster and cheaper. But a general small extraction-focused LLM would solve this.
I thought about fine-tuning Llama3-1B or Qwen models on larger models outputs, but my focus is currently elsewhere.
- msp26 2y agoHave you looked into structured generation with a library like outlines? https://github.com/dottxt-ai/outlines https://github.com/dottxt-ai/outlines
- druskacik 2y agoYes! This library is great and definitely helps, but I still had problems with performance. For example, smaller models would still hallucinate when extracting a JSON field when the given field wasn't present in the text (I'd expect null, but it provided either an incorrect value from the text, or a totally made up value).
- petercooper 2y agoJina did something related for extracting content from raw HTML and wrote about the techniques they used here: https://jina.ai/news/reader-lm-small-language-models-for-cleaning-and-converting-html-to-markdown/ https://jina.ai/news/reader-lm-small-language-models-for-cle... .. in my tests, the 1.5bn model works extremely well, though the open model is non commercial.