They are testing for a very straightforward needle retrieval, as LLMs traditionally were terrible for this in longer contexts. There are some more advanced tes

They are testing for a very straightforward needle retrieval, as LLMs traditionally were terrible for this in longer contexts. There are some more advanced tests where it's far less impressive. Just a couple of days ago Adobe released one such test- https://github.com/adobe-research/NoLiMa https://github.com/adobe-research/NoLiMa