2 ms·
Sounds like this would be expensive on the input token side with some saving on the decode side. For workloads with large documents / context that could be an i
by sarjann 3mo ago
Sounds like this would be expensive on the input token side with some saving on the decode side. For workloads with large documents / context that could be an issue.
- crazytweek 3mo agoAFAIK they solve this through their RAG based knowledgebases to only search and use the most relevant Information.