3 ms·
should be yes, but even in examples from openai, they usually do splitting into chunks For example in chatgpt-retrieval-plugin[0] repo default chunk size is ju
by egorr 3y ago
should be yes, but even in examples from openai, they usually do splitting into chunks
For example in chatgpt-retrieval-plugin[0] repo default chunk size is just 200 tokens
this is anyway a limitation, no doubt, but chunking is pretty often used
[0] https://github.com/openai/chatgpt-retrieval-plugin/blob/main/services/chunks.py https://github.com/openai/chatgpt-retrieval-plugin/blob/main...