Part of the problem IMO might be that OP is relying on OAI and Google's systems to dump data from websites and pdfs into the context and hoping it's correct and

Part of the problem IMO might be that OP is relying on OAI and Google's systems to dump data from websites and pdfs into the context and hoping it's correct and formatted properly. It probably is, but the odd random skips could also be explained by copy paste fails. Would be a better comparison if it was just provided in plain text. It's also all long context models getting pages of data, which even for these flagship ones is certainly just RoPE or similar which is a cheap hack but isn't super accurate [0]. 4o is best and still showing haystack benchmark accuracies below 80% and Gemini is just completely blind. That certainly needs fixing up to 100% before we can say for sure that nothing will ever get skipped. [0] https://preview.redd.it/rlgauej7ve4d1.jpeg?width=1086&format=pjpg&auto=webp&s=fa4e288f8f62c75523993aa4a15867d11889d508 https://preview.redd.it/rlgauej7ve4d1.jpeg?width=1086&format...