That's a great distinction to draw. I think it will split in two. There will be cases where the LLM has the truth represented in its data set and still chooses

That's a great distinction to draw. I think it will split in two. There will be cases where the LLM has the truth represented in its data set and still chooses to say something else because its training has told it to produce the most plausible sounding answer, not the one closest to the truth. So this will fit closer to the idea of real lying. A good example: I asked it what the differences in driving between Australia and New Zealand are. It confidently told me that in New Zealand you drive on the right hand side of the road while in Australia you drive on the left. I am sure it has the correct knowledge in its training data. It chose to tell me that because that is a more common answer people say when asked about driving differences because that is the more dominant difference when you look between different countries. Then there will be cases where the subject in question has never been represented in its data set. Here I think your point is very valid.