The problem with comparing LLM and human context length is that LLMs don't update their weights based on their input. The 32k of context that they do have is th

The problem with comparing LLM and human context length is that LLMs don't update their weights based on their input. The 32k of context that they do have is their only memory. Humans have multiple layers of memory and can recall things and concepts from years in the past - akin to millions of tokens' worth of recall in an LLM. Yes, that memory is extremely lossy, but it's there.