3 ms·
RAG tools are usually very trivial w.r.t. extracting clean data for embeddings. Demos typically show clean data sources with orthogonal data which lends itself
by hackernoteng 3y ago
RAG tools are usually very trivial w.r.t. extracting clean data for embeddings. Demos typically show clean data sources with orthogonal data which lends itself well to good performing embeddings based IR. Real-world data is mixed up, messy, and requires a lot of work to extract, clean, normalize, etc. Typically fine-tuned embedding model would be needed.
- CharlesW 3y ago> Real-world data is mixed up, messy, and requires a lot of work to extract, clean, normalize, etc. Typically fine-tuned embedding model would be needed. I'd think the former is required for whichever strategy you choose, although I haven't found it necessary to fine-tune a base LLM. What use case are you thinking of when you say, "typically fine-tuning is required"?