Large stone houses needed numbers of servants to collect and prepare firewood to keep them warm, and required materials (and workers) no long available to maint

Large stone houses needed numbers of servants to collect and prepare firewood to keep them warm, and required materials (and workers) no long available to maintain. Even the 'rich' kings and queens after the romans were no where near as cash rich as many of the romans that managed parts of the UK, as the population dropped significantly and slavery (while it still existed) wasn't as common as roman times. Plus we know building were used, for example as farm buildings - but it was more common for the building materials to be reused, rather than people moving into houses. I don't think they would have been too scared of the romans, as largely the lands were taken over by new groups - which the romans could not afford to keep defending against. There is a similar reason that amazing old houses lie for ages untouched in France - the cost to run and keep the is huge (and always has been) - but they no longer own the land/farms required to fund the buildings (and lives of the owners), so they sit rotting away.