2 ms·
In essence that's exactly the idea. It's not what you think it is though. It's choose the right words in the right places under the right context. You submit
by Zen_Sherbert 1y ago
In essence that's exactly the idea.
It's not what you think it is though. It's choose the right words in the right places under the right context.
You submit a 5 million token document of mixed data. It's a jumble of finances, cooking, and old stereo instructions for some reason.
You ask a the model what ingredients are in a chicken caprese.
It won't have to read millions of tokens, it will understand the what and the where and the why.
So chunking specifically isn't about understanding an entire context window of 5 million.
It's more about working with it in small pieces in relation to inference.
It is not a replacement, rather an alternative. An early one at that.
Thank you for taking the time to read, I appreciate the input and the skepticism too.
If you have more, can you provide any?
- gus_massa 1y ago> It's not what you think it is though. It's choose the right words in the right places under the right context. That's approximately what I thought. I want to be sure. Anyway, the details are very important. > I appreciate the input and the skepticism too. Let's say 20% skepticism and 80% it sounds like a good idea. I'm not using AI models too much, so it's hard for me to evaluate it. Let's hope someone else can give requested and unrequested feedback.