Sure your linear regression example is great, and it's an example of how 'hallucination' in models can be good. Imagine if the only prediction algorithms had to

Sure your linear regression example is great, and it's an example of how 'hallucination' in models can be good. Imagine if the only prediction algorithms had to fit every point in noisy data, it would look like either some overfitted thing like an ugly high order polynomial that goes through every point or just a database of every x y pair in the training set. Neither of those ones hallucinate if you ask the data from the training set. But for most purposes they wouldn't be as useful as a fitted linear regression, which 'hallucinates' y values even if you give it exact x from the training set. Of course if you just want to memorize and repeat back the training set then just a database is the best way!