5 ms·
I think this is just what you see with typical ISP "traffic shapers". They try to limit bandwidth to video sites, but since most video traffic is transferred b
by londons_explore 3y ago
I think this is just what you see with typical ISP "traffic shapers".
They try to limit bandwidth to video sites, but since most video traffic is transferred by HTTPS these days they end up just making a massive list of IP's which look like they might be sending video data and dropping some percentage of traffic to those IP's. Most CDN's are probably on the list.
End result is most video sites drop back to SD rather than HD.
If you do a speedtest, it will come out as fast. If you VPN, that will also be fast.
The IP range has nothing todo with it - it is the route the packets traverse and what the packets look like when they pass the shaper device that matters.
You could theoretically find out which device on the path is doing the dropping by manipulating the TTL of packets in a live TCP session and seeing when you get back TTL exceeded messages.
- supriyo-biswas 3y agoThere’s fast.com and speed.cloudflare.com to get realistic speed measurements since there’s no way* the ISP can tell traffic to these sites vs. video traffic to Netflix/Cloudflare. *Technically some ISPs can (and probably do) look at the ServerNameIndication to determine hostname and therefore whether they’re video flows or not, but I’ve not heard any real world examples of this being done.
- paulryanrogers 3y agoESNI to the rescue?
- cesarb 3y ago> ESNI to the rescue? IIRC, it's now called ECH, because AFAIK the focus changed from "encrypt the SNI" to "encrypt the whole Client Hello".
- crotchfire 3y agoUh, no, the focus changed to don't encrypt the outer SNI but pretend we're encrypting something that will prevent filtering, even though it won't.
- infotogivenm 3y agoCan you elaborate? I have not kept up with the eSNI/ECH stuff... How will filtering still be possible?
- lxgr 3y agoIsn't the outer SNI something that can be deduced from the IP address anyway?
- jacooper 3y agoDo you have more details on that ?
- avipars 3y agoComputer Scientists pick the best names for things
- bsoft16385 3y agoModern deep packet identification gear looks at packet size and pacing to distinguish video traffic patterns. It is possible to fool these systems, but it is not trivial.
- Waterluvian 3y agoAny reason fast.com couldn’t stream a video as the test content?
- pests 3y agoI think it does pull a real video, I'm not sure its pulled in the same way the actual Netflix video player would do it.
- tuetuopay 3y agoyou wouldn't pull at gigabits/s bandwidth when casually watching netflix
- londons_explore 3y agodunno about netflix, but youtube usually does. They send you a chunk of data as fast as possible, and then send nothing till your client requests the next chunk. By doing that, they can get a good estimate of your connections available bandwidth, which is needed for the decision whether to automatically switch to a higher/lower quality feed. It also means they can use 'dumb' servers which don't do any application-specific logic for throttling.
- tuetuopay 3y agomakes sense, but in that case it's quite easy to distinguish between a burst of traffic of <1s to a speedtest that last tens of seconds
- chatmasta 3y agoRecently I've noticed my phone disconnecting from my Wi-Fi router (a relatively underpowered device running OpenWRT and WireGuard) when starting YouTube videos. I guess this could explain it, if the router is hiccuping on that initial blast of data. I also use a Safari extension that lets me choose the YouTube video quality, so that might affect it if I'm not giving the feedback they expect for "auto" quality selection. (The issue is not the router, though - this doesn't happen on my laptop. I think my phone is just more aggressive in assuming the Wi-Fi is down and it's better to switch to cellular. But couple that with my phone's policy to use VPN on cellular, and the switch becomes much higher friction. I tried simply disabling cellular data, but then it's even worse because every time the phone disconnects from Wi-Fi it pops an alert telling me to enable cellular data.)
- crotchfire 3y agoAre you kidding? Snooping like this is EXACTLY why we still have unencrypted ServerNameIndication, even to this day in ECH (which still leaves the outer SNI in the clear).
- lxgr 3y agoWhat does the outer SNI tell a hypothetical filtering/traffic shaping ISP that the IP address doesn't already?
- deleted 3y ago[deleted]
- londons_explore 3y agolots of sites often just are connections to cloudflare/akami/fastly these days. Ie. one IP, millions of websites. The ISP/feds need SNI to tell what you're up to.
- lxgr 3y agoYes, but the outer SNI (with ECH) in that case would just say "Cloudflare", "Akamai" etc., if I understand it correctly.
- gruez 3y ago>but since most video traffic is transferred by HTTPS these days they end up just making a massive list of IP's which look like they might be sending video data and dropping some percentage of traffic to those IP's. You really don't when every modern browser sends SNI, which exposes the full domain name it's trying to connect to.
- karmakaze 3y agoIn most cases an ISP is also doing DNS resolution so would already be able to pretty accurately reverse map IPs to hostnames without reverse records.
- lxgr 3y agoThat's much easier to fix (by using DoH or DoT) than getting ECH widely deployed, though. Importantly, it can be done unilaterally by each user and doesn't need cooperation on both sides of the connection. Many browsers these days support using DoH without changing any OS settings.
- karmakaze 3y agoThat's not actually a fix. As long as some other subscriber does the DNS lookup, then the ip address will be associated with the name that was looked up. If you happen to use a different resolver and get the same address, there's no difference.
- lxgr 3y agoIf the IP address belongs to a cloud load balancer or CDN, that’s still much better than hostname-level detail. Even for subdomains it helps (think somebody.bloghoster.tld). And ECH will still reveal the outer SNI, apparently.
- guappa 3y agoI got called a paranoid a couple of years ago, because I said that my provider slows down long downloads, since every download starts fast, but after a few megabytes they become slower and slower.
- lxgr 3y agoAre you using cable by any chance? This is a pretty standard feature of DOCSIS, I believe. It allows bursting above the provisioned connection speed for a few seconds when there's capacity on the shared network segment.
- callalex 3y agoThat’s not paranoia, it’s bog standard traffic shaping that ISPs have been doing since the 90’s or even earlier.
- guappa 3y agoEh, others in the discussion claimed it would be completely impossible for an ISP to do this thing, due to how network protocols work. Anyway, annoying… at least http supports resuming, so it is possible to kill and restart manually, or use aria2c to circumvent it.
- kaetemi 3y agoISP controlled modem counts your incoming data in a short rolling window. If it exceeds a value, it throttles below your normal speed and buffers or drops incoming packets to slow things down, until the bandwidth budget is compensated for (and the dropped packets are all counted, so it could be stuck there longer than anticipated), then switches to normal speed until traffic goes below a threshold, then switches back to full speed burst. Worst case the flow control reacts slower than the modem burst switching and you end up in an oscillating pattern. The trick is just to do your own rate limiting without burst at the right limit, that bypasses whatever shitty burst implementation your ISP modem has and weird shit won't happen anymore.
- kaetemi 3y ago
- bobdvb 3y agoIt doesn't have to be traffic shaping, it can be something as simple as limited peering capacity. Hanlon's Razor applies, this isn't necessarily malicious capping. Most ISPs don't have one fat pipe to this magical "internet" place, they have multiple pipes going different places and not all of them are even listed in peeringdb, so their route to a particular CDN might be over private peering which is limited in size (e.g. 1-10Gb). Sure, then excess traffic might go over public peering but that might also then get saturated, or the ISP might force traffic over the private peering to keep the public peering clear for general traffic. If you use a VPN then the traffic will come from a different route and different peering, thus be subject to the capacity of that route. What the OP probably wants is for Zen to have better peering with Akamai (they already have some announced routes).