4 ms·

Note that some ISPs prioritise traffic to known speedtest targets, turning off traffic shaping rules that might otherwise slow bulk transfers. When this happens

Note that some ISPs prioritise traffic to known speedtest targets, turning off traffic shaping rules that might otherwise slow bulk transfers. When this happens it means you are testing the likely maximum throughput of your connection not necessarily the throughput you will see more generally. This is why Netflix started fast.com - because it draws data from the same distribution points as their video streaming apps it means you can't prioritise the speedtest without also doing so for the video traffic or (more likely) you can't de-prioritise the video traffic without also getting bad scores in that particular speedtest. From Netflix's point of view it is an answer to people contacting support with "my speedtest results are fine, the problem must be your servers" when they are experiencing video lag/drops and other such problems and the issue is due to ISP traffic shaping or the ISP simply not having enough backhaul bandwidth. A more reliable test might be taking part in a busy public torrent: that way you are testing against arbitrary locations so your ISP can't be setting different shaping rules for them. Just remember to throttle upstream when testing downstream and vice-versa or saturation in the other direction will slow control packets that will in turn give you lower results for the one you are testing. This may fall into another trap though: unless you limit the number of active streams it may be an unrealistic test as more generally most processes use a small number of streams (or just a single one), and if you limit the number of streams too much you might get a lower result because each swarm member you connect to may be fairly saturated and sharing its bandwidth amongst many connections.

jermaustin1 7y ago
That said, I can and do get 200mbps+ on fast.com (my preferred/goto speedtest), and still experience constant buffering and the "sorry we can't play that video right now" message that kicks me out but then just clicking play again works. I'm not saying it's their servers or network, but I haven't found a reason for it based on my fast.com results.
- HerrFolgreich 7y ago
  > but then just clicking pay again works freudian slip much?
  - jermaustin1 7y ago
    Good catch!
- dspillett 7y ago
  It may be that your ISP, or another organisation that your traffic flows through, has implemented an efficient and accurate-enough way to identify speed-test traffic from amongst the actual video traffic. Possibly just by assuming that after a few minutes of transfer or a certain amount of traffic, your current use is not likely to be just for speed testing. I assume this is why the tester does not run an upstream test by default: otherwise the lack of a block of upstream traffic would be a good signal that video playing is going on rather than throughput testing.
  - jermaustin1 7y ago
    Possibly. I wouldn't put anything past optimum after they started injecting advertisements for new channels and packages on my non-https traffic.
PeterisP 7y ago
A busy public torrent is a bad predictor for me because my main bottleneck for most traffic seems to be after my ISP (i.e. their uplink or their international connection - possibly due to shaping and not a hard limit) not the 'last mile', so if a torrent has a single peer that's local then I can get multiple times larger speed than usually. Though with fiber in most cases speed can be ignored, as it's simply "enough" or limited by the server capacity on the other side.
- dspillett 7y ago
  Another option would be to spin up a few VMs on various pay-per-use cloud providers, perhaps VMs in a couple of Azure regions, and test by pulling data from those. The variety of regions will rule out local traffic problems at the other end, the pay-by-hour/minute/second nature of such VMs would make it a cheap test to run. Especially if you can leave the VMs configured but stopped between tests and have no cost in that state, so you pay nothing most of the time but don't have to wait for provisioning & configuration each time. Or if you have external servers for other uses, and/or have friends who do, you could use those instead of needing to spin up anything new.
chrisweekly 7y ago
IME the best way to do it is to configure a WPT (WebpageTest) instance with geo-distributed clients, and trigger test runs from your local terminal.
mholt 7y ago
My free Google Fiber connection (5 Mbps down) always scored ~100 Mbit speeds at fast.com, but was 5 Mbps on speedtest.net. https://twitter.com/mholt6/status/999425756198387713 https://twitter.com/mholt6/status/999425756198387713
- leowinterde 7y ago
  So Google Fiber is throttling your connection, as you paid for, but the network connection to netflix is unthrottled. Netflix is also deploying OCA (Open Connect Appliance) servers directly inside most big ISPs - https://openconnect.netflix.com/en/#sample-architectures https://openconnect.netflix.com/en/#sample-architectures
- dspillett 7y ago
  I've seen similar on a 100mbit line that was actually a faster line throttled to that speed, the effect was particularly upstream rather than down. Essentially some content (or more likely content from some locations) is allowed to bypass the throttle. This might be deliberate but not malicious: through Netflix's Open Connect program ISPs can host a local cache of Netflix content to reduce their peering costs while still offering full service to many concurrent users. If the main throttling effect you are experiencing is applied topologically close to their peering lines such that it would apply to the Open Connect equipment too, rather that the Open Connect kit setting between the throttle point and the peering lines, then to make it useful in the case of new or otherwise not recently accessed data the traffic shaping rules would need to let Netflix traffic (including your speedtests) through relatively unhindered. The path between the Open Connect equipment and your line is unlikely to need throttling because that will be an internal matter with very different cost dynamics than external peering.
chrissnell 7y ago
I wrote my own console-based client and server to get around this problem. It uses its own protocol on its own ports and runs on servers that you control. https://github.com/chrissnell/sparkyfish https://github.com/chrissnell/sparkyfish

jermaustin1 7y ago

That said, I can and do get 200mbps+ on fast.com (my preferred/goto speedtest), and still experience constant buffering and the "sorry we can't play that video right now" message that kicks me out but then just clicking play again works. I'm not saying it's their servers or network, but I haven't found a reason for it based on my fast.com results.

HerrFolgreich 7y ago
> but then just clicking pay again works freudian slip much?
- jermaustin1 7y ago
  Good catch!
dspillett 7y ago
It may be that your ISP, or another organisation that your traffic flows through, has implemented an efficient and accurate-enough way to identify speed-test traffic from amongst the actual video traffic. Possibly just by assuming that after a few minutes of transfer or a certain amount of traffic, your current use is not likely to be just for speed testing. I assume this is why the tester does not run an upstream test by default: otherwise the lack of a block of upstream traffic would be a good signal that video playing is going on rather than throughput testing.
- jermaustin1 7y ago
  Possibly. I wouldn't put anything past optimum after they started injecting advertisements for new channels and packages on my non-https traffic.

PeterisP 7y ago

A busy public torrent is a bad predictor for me because my main bottleneck for most traffic seems to be after my ISP (i.e. their uplink or their international connection - possibly due to shaping and not a hard limit) not the 'last mile', so if a torrent has a single peer that's local then I can get multiple times larger speed than usually. Though with fiber in most cases speed can be ignored, as it's simply "enough" or limited by the server capacity on the other side.

dspillett 7y ago
Another option would be to spin up a few VMs on various pay-per-use cloud providers, perhaps VMs in a couple of Azure regions, and test by pulling data from those. The variety of regions will rule out local traffic problems at the other end, the pay-by-hour/minute/second nature of such VMs would make it a cheap test to run. Especially if you can leave the VMs configured but stopped between tests and have no cost in that state, so you pay nothing most of the time but don't have to wait for provisioning & configuration each time. Or if you have external servers for other uses, and/or have friends who do, you could use those instead of needing to spin up anything new.

chrisweekly 7y ago

IME the best way to do it is to configure a WPT (WebpageTest) instance with geo-distributed clients, and trigger test runs from your local terminal.

mholt 7y ago

My free Google Fiber connection (5 Mbps down) always scored ~100 Mbit speeds at fast.com, but was 5 Mbps on speedtest.net. https://twitter.com/mholt6/status/999425756198387713 https://twitter.com/mholt6/status/999425756198387713

leowinterde 7y ago
So Google Fiber is throttling your connection, as you paid for, but the network connection to netflix is unthrottled. Netflix is also deploying OCA (Open Connect Appliance) servers directly inside most big ISPs - https://openconnect.netflix.com/en/#sample-architectures https://openconnect.netflix.com/en/#sample-architectures
dspillett 7y ago
I've seen similar on a 100mbit line that was actually a faster line throttled to that speed, the effect was particularly upstream rather than down. Essentially some content (or more likely content from some locations) is allowed to bypass the throttle. This might be deliberate but not malicious: through Netflix's Open Connect program ISPs can host a local cache of Netflix content to reduce their peering costs while still offering full service to many concurrent users. If the main throttling effect you are experiencing is applied topologically close to their peering lines such that it would apply to the Open Connect equipment too, rather that the Open Connect kit setting between the throttle point and the peering lines, then to make it useful in the case of new or otherwise not recently accessed data the traffic shaping rules would need to let Netflix traffic (including your speedtests) through relatively unhindered. The path between the Open Connect equipment and your line is unlikely to need throttling because that will be an internal matter with very different cost dynamics than external peering.

chrissnell 7y ago

I wrote my own console-based client and server to get around this problem. It uses its own protocol on its own ports and runs on servers that you control. https://github.com/chrissnell/sparkyfish https://github.com/chrissnell/sparkyfish