One interesting proposal here is a multiple NIAH retrieval benchmark. When they put 100 needles, then the recall rate becomes considerably lower, something arou

One interesting proposal here is a multiple NIAH retrieval benchmark. When they put 100 needles, then the recall rate becomes considerably lower, something around 60~70%. Not sure what's the exact configuration of this benchmark, but intuitively this makes sense and should be a critical metric for the model's reliability.