I think the point is only 'locally' right. If you consider a long enough period of time, effectively no one has ever been able to afford the luxury of 'building

I think the point is only 'locally' right. If you consider a long enough period of time, effectively no one has ever been able to afford the luxury of 'building a toy'. And yet, at least some people do now have that luxury. It certainly isn't impossible for lots and lots of people to afford some degree of something pretty similar tho.