> Future LLMs, regardless of model architecture, are fundamentally constrained in this way, by virtue of the fact that they are trained on human-written texts.

> Future LLMs, regardless of model architecture, are fundamentally constrained in this way, by virtue of the fact that they are trained on human-written texts. Therefore, the outputs of a LLM, at best, reflect our current understanding of the universe and nothing more. That constraint holds only for pure LLMs. There is a secondary source of learning - external feedback to the model. When a LLM is part of a larger system, it can integrate feedback to improve itself, like AlphaGo Zero. A LLM+compiler could self-optimize by observing compilation errors and program outputs. A LLM+game could self-improve by conditioning actions on the score. Humans could create a preference dataset and model for RLHF to refine the LLM based on human feedback. A LLM+robot could learn from past experiences to improve its planning. Similarly, humans need tools to conduct research. We can't do pure research without labs and experiments. LLMs need external confirmation the same way we do. The outputs of a pure LLM merely reflect our current understanding of the universe. But coupled with real-world feedback, LLMs have the potential to learn and discover new knowledge.