GPT 4 already has 32k tokens of context for prompts. Once we’re making arguments about scale only a few orders of magnitude larger than the current state of the

GPT 4 already has 32k tokens of context for prompts. Once we’re making arguments about scale only a few orders of magnitude larger than the current state of the art, it seems similar to arguments 10-15 years ago that real-time ray tracing is not feasible.