3 ms·
LlamaIndex can't do chunk-level metadata, only document-level metadata, so you can't put precise references to where materials the LLM synthesized answers from
by treprinum 3y ago
LlamaIndex can't do chunk-level metadata, only document-level metadata, so you can't put precise references to where materials the LLM synthesized answers from originated, e.g. HTML anchors. Just write your own RAG with Pinecone and OpenAI APIs directly.
- seldo 3y agoLlamaIndex lets you attach metadata to Nodes which are basically chunks, although that fact is poorly documented! Will fix.
- treprinum 3y agoThanks! Even with a better documentation, document importers don't extract node metadata so one needs to write their own "text and metadata extractor" as well. It's then easier to skip LlamaIndex altogether, or just get inspiration from some re-ranking etc. you guys did.
- anjanb 3y agocan you elaborate please