I think the reason is that the Culture has been around long enough to attain a level of technological development at which most civilizations sublime and simply

I think the reason is that the Culture has been around long enough to attain a level of technological development at which most civilizations sublime and simply stop participating in what we call material existence. The Culture could do so but has chosen not to do so (at least not collectively; individual people and minds sublime on a regular basis). A lot of the civilizations that would be militarily or economically more powerful than the culture aren't a problem because they've already sublimed.