2 ms·
Context window size will eventually be solved, likely with its own trade offs.
by azinman2 3y ago
Context window size will eventually be solved, likely with its own trade offs.
- binarymax 3y agoThe way to think about it is this: you can scan an entire book for the part you’re looking for (a huge context window), or you can look it up in the index in the back (a good retriever). The latter is a better approach when you’re serving production use cases. It’s faster and less expensive.