There wouldn't necessarily be an extreme event like a famine in which only consuming milk would save you. It could well be that consuming milk gave you even jus

There wouldn't necessarily be an extreme event like a famine in which only consuming milk would save you. It could well be that consuming milk gave you even just 1% more calories on average (not skipping a meal every so often, when all there is is milk to be eaten right at that moment, because other food spoiled or was unavailable). Even a 1% fitness increase compounds itself over multiple generations into a large effect, even if in each generation it is so small it is hardly noticeable (and so not written into legend).