The question is, how do you know how much is to be gained, unless you try? Just because you see most customers being on high speed connections most of the time

The question is, how do you know how much is to be gained, unless you try? Just because you see most customers being on high speed connections most of the time doesn't mean much: for one, it could be because otherwise it's so bad no one does it; plus, a better experience for slow connections/older devices is quite likely to translate to better experience for everyone. Otherwise, you might figure it out when you suddenly lose most of your users to a competitor who offers a snappier experience.