The problem with those numbers is they hit the internal limit before you use all those tokens. There's a limit to how many rules or factors their conditional pr

The problem with those numbers is they hit the internal limit before you use all those tokens. There's a limit to how many rules or factors their conditional probability model can keep track of. Once you hit that having a bigger context window doesn't matter.