I think it’s pretty clear by now that LLMs are what the name says: language models. Text completion engines. If the question you pose to an LLM is similar to q

I think it’s pretty clear by now that LLMs are what the name says: language models. Text completion engines. If the question you pose to an LLM is similar to questions it’s been trained on, there’s a good chance you’ll get something useful back. If the question you ask is novel, you’re more likely to get gibberish. This is really no different than interpolating vs. extrapolating using traditional statistical models. Keep in mind that the only reason LLMs answer questions at all is that they’ve been fine-tuned on extra question-and-answer formatted texts. Otherwise you would need to prompt the question so that the answer is a natural continuation of the text — based on the types of text in the pretraining data.