3 ms·

Could speed test sites do something about this? Like obfuscate the traffic somehow to get more genuine results?

bfz 4y ago
It's possible to build a speed test site out of media file fetches from otherwise public assets from third party sites, the quality of the measurement will just be a little noisier. In many ways this would actually be a better test than some well-placed IP address, since there is a chance to randomly sample many different routes from the ISP to real world destinations for practical object sizes (i.e., not just an idealized unidirectional stream with no request roundtrips in between). For example you could do something with Image.onload event, but even better are the recent JS web performance APIs, they provide a huge amount of detail - including DNS lookup latency, TCP connect time, TTFB etc. Actually now I want to build this edit: I'm probably not going to build this, but you should. For large object sources, there are Flickr full size images, HLS URLs for video on demand systems (each HLS segment is generally around 1-3MB), etc. I'd want a giant list of these to pick from at random, and probably want to bucket the results along (domain name, destination BGP ASN, destination GeoIP) axes as well as whatever the JS performance API offers. You could maybe present each axis as some kind of confidence interval, then geometric mean to produce an overall score presented graphically somehow You could then collect up all these scores and publish stats bucketed by Internet provider. And that's the point where I realized I don't have the energy to build all this ;)
- canadaduane 4y ago
  This sounds so much closer to a "real" test than our single-server tests.
- GoOnThenDoTell 4y ago
  This sounds awesome!
Klathmon 4y ago
I believe this is the idea behind the https://fast.com https://fast.com speed test. It just downloads data from Netflix servers, so the results you get are basically guaranteed to be the speeds you would get with Netflix.
- colinmhayes 4y ago
  > I believe this is the idea behind the https://fast.com https://fast.com speed test. Or maybe the idea was that Netflix would get the speedtest treatment.
vitus 4y ago
This is one of the biggest selling points of fast.com -- it's served from the same Netflix domain + servers that also stream video (nflxvideo.net). If ISPs want to make fast.com fast, then they need to do so for all of Netflix.
manicpolymath 4y ago
I think the bigger problem isn’t so much the traffic type as it is traffic destination. As long as there are only a few hosts or domains providing speed test services (and there always will be: upload speeds are expensive) ISPs will be able to whack-a-mole with their whitelists. I’d say that this needs some sort of regulation, but as long as ISPs are the gatekeepers, they can cheese $Government all day long too. This word gets overused perhaps, but the closest I can come up with is a decentralized monitoring setup with random speed test hosts (especially hosted @home style). Care would have to be taken to avoid how-are-these-still-legal data caps though.

bfz 4y ago

It's possible to build a speed test site out of media file fetches from otherwise public assets from third party sites, the quality of the measurement will just be a little noisier. In many ways this would actually be a better test than some well-placed IP address, since there is a chance to randomly sample many different routes from the ISP to real world destinations for practical object sizes (i.e., not just an idealized unidirectional stream with no request roundtrips in between). For example you could do something with Image.onload event, but even better are the recent JS web performance APIs, they provide a huge amount of detail - including DNS lookup latency, TCP connect time, TTFB etc. Actually now I want to build this edit: I'm probably not going to build this, but you should. For large object sources, there are Flickr full size images, HLS URLs for video on demand systems (each HLS segment is generally around 1-3MB), etc. I'd want a giant list of these to pick from at random, and probably want to bucket the results along (domain name, destination BGP ASN, destination GeoIP) axes as well as whatever the JS performance API offers. You could maybe present each axis as some kind of confidence interval, then geometric mean to produce an overall score presented graphically somehow You could then collect up all these scores and publish stats bucketed by Internet provider. And that's the point where I realized I don't have the energy to build all this ;)

canadaduane 4y ago
This sounds so much closer to a "real" test than our single-server tests.
GoOnThenDoTell 4y ago
This sounds awesome!

Klathmon 4y ago

I believe this is the idea behind the https://fast.com https://fast.com speed test. It just downloads data from Netflix servers, so the results you get are basically guaranteed to be the speeds you would get with Netflix.

colinmhayes 4y ago
> I believe this is the idea behind the https://fast.com https://fast.com speed test. Or maybe the idea was that Netflix would get the speedtest treatment.

vitus 4y ago

This is one of the biggest selling points of fast.com -- it's served from the same Netflix domain + servers that also stream video (nflxvideo.net). If ISPs want to make fast.com fast, then they need to do so for all of Netflix.

manicpolymath 4y ago

I think the bigger problem isn’t so much the traffic type as it is traffic destination. As long as there are only a few hosts or domains providing speed test services (and there always will be: upload speeds are expensive) ISPs will be able to whack-a-mole with their whitelists. I’d say that this needs some sort of regulation, but as long as ISPs are the gatekeepers, they can cheese $Government all day long too. This word gets overused perhaps, but the closest I can come up with is a decentralized monitoring setup with random speed test hosts (especially hosted @home style). Care would have to be taken to avoid how-are-these-still-legal data caps though.