7 ms·
I love that this exists but I worry how it uses the term “train”, even in quotes, as I spend a lot of time explaining how RAG works and I try to emphasize that
by pamelafox 3y ago
I love that this exists but I worry how it uses the term “train”, even in quotes, as I spend a lot of time explaining how RAG works and I try to emphasize that there is no training/fine-tuning involved. Just data preparation, chunking and vectorization as needed.
- zainhoda 3y agoAuthor of the package here. I would be open to alternative suggestions on terminology! The problem is that our typical user has never encountered RAG before.
- pamelafox 3y agoHm we typically talk about that phase as “data ingestion”. We also use verbs extract, chunk, index, but those are parts of ingestion. Another phrase we use is “data preparation”. The tricky thing is that your train() method is on the vanna object, so if you called it “ingest”, it wouldn’t be clear where the data was headed. But I think it’d be clearer than “train” given that has such a specific meaning in this space. You could also look to LlamaIndex API for naming inspiration.
- zacmps 3y agoI've been using 'build' as in, 'build a dataset'. I think it gets the same idea across without being as easy to confuse with fine-tuning.