3 ms·
Does Opus 4.6 find the needle in the haystack?
- grey-area 8mo agoA recall rate of 0% for information outside the original training data in this experiment.
- amai 8mo ago„It's not magic just yet, models aren't purely performing context-based retrieval. They're leveraging their pre-training knowledge extensively. Always be careful when running retrieval experiments on items that might be well represented in the training data (read: the entire public Internet). Results for never before seen data might be much less impressive.“ This is a really important message regarding the power of LLMs.