4 ms·

> simply hard-linked to the resources It's fairly surprising to see that approach from pretty large brands/companies. Some Indian ISPs are kinda notorious in t

> simply hard-linked to the resources It's fairly surprising to see that approach from pretty large brands/companies. Some Indian ISPs are kinda notorious in that they'll just link to huge image files on third-party sites and just let their 300 million customers hammer the poor site into the ground. I guess it saves them bandwidth, but they also run a huge risk of pissing someone off and have that asset replaced by something nasty. When people use this kind of hard-linking/hot-linking of resource I don't think they do it to save money or to be evil, I think it mostly gross incompetence. We had an issue where our product images would just be ruthlessly scraped, if there had been some rate-limiting in place we'd most likely just have allowed it or not noticed. Normally I just pointed the poor scraper to a multi-gigabyte file or funny and unrelated image, if you want to download the same image of a beaver 25.000 times go ahead. But I also seen amateurishly search engines just pound a site to the point where it was easier to just deny all traffic coming from their crawler, as compared to try to reach out and figure something out. Say what you will about Google and Bing, their crawlers are well written and well behaved.

ecshafer 3y ago
The isps can pretty easily set up a cache for things that are popular. There was a time when I worked for an ISP consultancy, where Netflix would literally give ISPs a prebuilt cache server iirc for free, and that was a win win for everyone. I shouldn’t be surprised that some shitty isp thinks it’s easier to hard link, but making a cache isn’t that hard.
- abdullahkhalids 3y ago
  A couple of years ago, the main submarine link connecting my country to the internet was damaged, and internet basically slowed to a crawl for a day. Could barely open anything. But Netflix was running at full HD with no problem. So they must have had local cache's with the ISPs.
  - masukomi 3y ago
    this is how CDNs make their money. They replicate popular content close to the users. They generally do that WITHOUT giving anyone hardware, but by renting hardware in smaller areas, and setting up their own data centers in popular areas. source: I used to work at Akamai (it was a while ago). The myriad things they do with their edge servers is pretty amazing, but for simple CDN stuff, most of the time clients aren't shipping hardware to isps. That being said, if Netflix can convince an isp to pay its electricity and hardware maintenance costs for a popular isp's customers that's going to pay for itself very quickly. So, i can see why Netflix offers this. It's just not normal AFAIK.
  - andylynch 3y ago
    Very likely. Netflix has a program where they ship edge cache appliances to ISPs free of charge.
    - Lukas_Skywalker 3y ago
      There are some public details about the program, and it's quite interesting: https://openconnect.netflix.com/en/appliances/ https://openconnect.netflix.com/en/appliances/
    - gorlilla 3y ago
      I recall when they approached Verizon for the same thing; Verizon wanted rent[0]. [0]: https://time.com/2866004/verizon-netflix/ https://time.com/2866004/verizon-netflix/
- bombcar 3y ago
  I think they're referring to the famous Wikimedia flower - https://phabricator.wikimedia.org/T273741 https://phabricator.wikimedia.org/T273741 And it's much harder now to have a caching server because everything is HTTPS, so you have to coordinate with both ends to make it work.
Logans_Run 3y ago
I'm not defending the practice by any stretch of the imagination (it's wrong to hotlink, simple as) but I can remember in the heady halcyon days of the early internet (back when Windows 95 had yet to become a thing) hotlinking was done out of pure ignorance about the impact it had on the hosting site and most doing it knew no better nor of the impact it had on the hot-linked servers' resources. Given that Eternal September remains a thing to this day, I wonder if that remains the case? As for Google Bot being well behaved - I'm glad to hear that this has changed because at the turn of the century (which is the last time I dipped my toe in to self hosting) it certainly wasn't well behaved in the slightest and it would ruthlessly crawl away, happily ignoring any robots.txt limitations applied.
LordDragonfang 3y ago
Yeah, if it were someone as important as an ISP doing it, you could easily be very petty and change it to an image that read "<isp> officially endorses <toxic political view>" (say, the statehood of Taiwan)
triniTY12Tree 3y ago
[flagged]

ecshafer 3y ago

The isps can pretty easily set up a cache for things that are popular. There was a time when I worked for an ISP consultancy, where Netflix would literally give ISPs a prebuilt cache server iirc for free, and that was a win win for everyone. I shouldn’t be surprised that some shitty isp thinks it’s easier to hard link, but making a cache isn’t that hard.

abdullahkhalids 3y ago
A couple of years ago, the main submarine link connecting my country to the internet was damaged, and internet basically slowed to a crawl for a day. Could barely open anything. But Netflix was running at full HD with no problem. So they must have had local cache's with the ISPs.
- masukomi 3y ago
  this is how CDNs make their money. They replicate popular content close to the users. They generally do that WITHOUT giving anyone hardware, but by renting hardware in smaller areas, and setting up their own data centers in popular areas. source: I used to work at Akamai (it was a while ago). The myriad things they do with their edge servers is pretty amazing, but for simple CDN stuff, most of the time clients aren't shipping hardware to isps. That being said, if Netflix can convince an isp to pay its electricity and hardware maintenance costs for a popular isp's customers that's going to pay for itself very quickly. So, i can see why Netflix offers this. It's just not normal AFAIK.
- andylynch 3y ago
  Very likely. Netflix has a program where they ship edge cache appliances to ISPs free of charge.
  - Lukas_Skywalker 3y ago
    There are some public details about the program, and it's quite interesting: https://openconnect.netflix.com/en/appliances/ https://openconnect.netflix.com/en/appliances/
  - gorlilla 3y ago
    I recall when they approached Verizon for the same thing; Verizon wanted rent[0]. [0]: https://time.com/2866004/verizon-netflix/ https://time.com/2866004/verizon-netflix/
bombcar 3y ago
I think they're referring to the famous Wikimedia flower - https://phabricator.wikimedia.org/T273741 https://phabricator.wikimedia.org/T273741 And it's much harder now to have a caching server because everything is HTTPS, so you have to coordinate with both ends to make it work.

Logans_Run 3y ago

I'm not defending the practice by any stretch of the imagination (it's wrong to hotlink, simple as) but I can remember in the heady halcyon days of the early internet (back when Windows 95 had yet to become a thing) hotlinking was done out of pure ignorance about the impact it had on the hosting site and most doing it knew no better nor of the impact it had on the hot-linked servers' resources. Given that Eternal September remains a thing to this day, I wonder if that remains the case? As for Google Bot being well behaved - I'm glad to hear that this has changed because at the turn of the century (which is the last time I dipped my toe in to self hosting) it certainly wasn't well behaved in the slightest and it would ruthlessly crawl away, happily ignoring any robots.txt limitations applied.

LordDragonfang 3y ago

Yeah, if it were someone as important as an ISP doing it, you could easily be very petty and change it to an image that read "<isp> officially endorses <toxic political view>" (say, the statehood of Taiwan)

triniTY12Tree 3y ago

[flagged]