That's because when they say "long context window" they're lying and they actually mean that they support a long input prompt that is still compressed into a sm

That's because when they say "long context window" they're lying and they actually mean that they support a long input prompt that is still compressed into a small context window. (Typically by throwing out tokens in the middle.) An actually large context window is impossible due to how LLM attention works under the hood.