He might have spent the money in ways that later benefited him - even if he bought a house that was 13$ more expensive, given the housing boom during his lifeti

He might have spent the money in ways that later benefited him - even if he bought a house that was 13$ more expensive, given the housing boom during his lifetime in many locations he would have made more than a 6% return. You're never going to be able to really figure that what-if game anyhow... I think interest is an appropriate simplification (certainly more appropriate than inflation).