5 ms·

In addition to being so small (text-based, minimal JS), HN is cached by CloudFlare, which grabs copies of dynamic pages and pushes those copies out to their CDN

In addition to being so small (text-based, minimal JS), HN is cached by CloudFlare, which grabs copies of dynamic pages and pushes those copies out to their CDN edge servers around the globe so they're closer to end users. When your browser asks for the IP to news.ycombinator.com, you get the address of a cloudflare CDN server. Which web server the DNS server will send you is based on a GeoIP lookup of your own IP address so that they can give you the address to the server physically closest to you, and therefore the lowest possible latency. Edit: Apparently this is not how CF does things (see below explanation by benjojo12) but is not uncommon in load-balancing and distribution. news.ycombinator.com. 296 IN CNAME news.ycombinator.com.cdn.cloudflare.net. news.ycombinator.com.cdn.cloudflare.net. 299 IN A 198.41.190.47 news.ycombinator.com.cdn.cloudflare.net. 299 IN A 198.41.191.47

benjojo12 11y ago
Pretty close! We don't actually use GeoIP at the DNS level, we use the routing table of the internet to direct people to the closest location we have to them by announcing the same IP addresses in the routing table in many places (eg, anycast) You can find a blog post primer and general more explanation here: https://blog.cloudflare.com/cloudflares-architecture-eliminating-single-p/ https://blog.cloudflare.com/cloudflares-architecture-elimina... the upshot of this is that TCP and SSL connections start faster because of lower latency to our location, and we can pull items out of cache faster and give them to the client.
- andrewstuart2 11y ago
  Awesome, thanks for the additional info. On a related note, I really appreciate how good CloudFlare is about knowledge sharing. Not a lot of companies would be so willing to share information that's so integral to their business.
  - Hello71 11y ago
    although it's useful to note that use of anycast is impossible to hide. any looking glass will tell you that CF has tons of routes to each apparent IP, e.g. https://ssp.pme.gin.ntt.net/lg/lg.cgi?query=bgp&sourceIP=IP&addr=198.41.191.47&router=Ashburn,+VA+-+US https://ssp.pme.gin.ntt.net/lg/lg.cgi?query=bgp&sourceIP=IP&...
  - fapjacks 11y ago
    On the other hand, they do some stupid stuff, disregarding the standard and how useful it is [0]. [0] https://blog.cloudflare.com/deprecating-dns-any-meta-query-type/ https://blog.cloudflare.com/deprecating-dns-any-meta-query-t...
    - MichaelGG 11y ago
      Disregarding standards is not uncommon when you're actually running a high-performance system that faces attacks. Plenty of stuff goes into specs that just doesn't make sense while operating. Their explanation of why they are disabling ANY seems perfectly fine.
      - fapjacks 11y ago
        All I can see is that you've never needed to debug problems with DNS.
jonahx 11y ago
So every time a new comment is posted, is the CloudFlare cache invalidated and replaced with a new version of the comments page?
- benjojo12 11y ago
  Actually for now, HN is not caching their HTML with us. we put a "CF-Cache-Status" header for when we are caching things
  - jonahx 11y ago
    So when hitting a comment page, HN's backend is being hit in some way, whether by reading a db or by concatenating flat files or something else? Or are they just using a local cache that isn't pushed out to CloudFlare because there would be so much churn that pushing to CF wouldn't be a benefit?
  - MichaelGG 11y ago
    Are they using Railgun to only transmit changes? Railgun seems like the neatest thing CF offers, as, AFAIK, you get the bandwidth benefits of caching without having to deal with invalidation.
- Alex3917 11y ago
  Not sure, but cloudfare supposedly speeds up content delivery even for dynamic content where you need a new db hit each time. The idea is that it leaves open a persistent connection with the webserver, which means A) less work creating and destroying connections B) each new connection isn't starting off as throttled.
  - logicuce 11y ago
    They have tech to do even better. See here https://www.cloudflare.com/railgun https://www.cloudflare.com/railgun That is how you can make it faster even when somebody is logged in and you are delivering a somewhat personalized experience.

benjojo12 11y ago

Pretty close! We don't actually use GeoIP at the DNS level, we use the routing table of the internet to direct people to the closest location we have to them by announcing the same IP addresses in the routing table in many places (eg, anycast) You can find a blog post primer and general more explanation here: https://blog.cloudflare.com/cloudflares-architecture-eliminating-single-p/ https://blog.cloudflare.com/cloudflares-architecture-elimina... the upshot of this is that TCP and SSL connections start faster because of lower latency to our location, and we can pull items out of cache faster and give them to the client.

andrewstuart2 11y ago
Awesome, thanks for the additional info. On a related note, I really appreciate how good CloudFlare is about knowledge sharing. Not a lot of companies would be so willing to share information that's so integral to their business.
- Hello71 11y ago
  although it's useful to note that use of anycast is impossible to hide. any looking glass will tell you that CF has tons of routes to each apparent IP, e.g. https://ssp.pme.gin.ntt.net/lg/lg.cgi?query=bgp&sourceIP=IP&addr=198.41.191.47&router=Ashburn,+VA+-+US https://ssp.pme.gin.ntt.net/lg/lg.cgi?query=bgp&sourceIP=IP&...
- fapjacks 11y ago
  On the other hand, they do some stupid stuff, disregarding the standard and how useful it is [0]. [0] https://blog.cloudflare.com/deprecating-dns-any-meta-query-type/ https://blog.cloudflare.com/deprecating-dns-any-meta-query-t...
  - MichaelGG 11y ago
    Disregarding standards is not uncommon when you're actually running a high-performance system that faces attacks. Plenty of stuff goes into specs that just doesn't make sense while operating. Their explanation of why they are disabling ANY seems perfectly fine.
    - fapjacks 11y ago
      All I can see is that you've never needed to debug problems with DNS.

jonahx 11y ago

So every time a new comment is posted, is the CloudFlare cache invalidated and replaced with a new version of the comments page?

benjojo12 11y ago
Actually for now, HN is not caching their HTML with us. we put a "CF-Cache-Status" header for when we are caching things
- jonahx 11y ago
  So when hitting a comment page, HN's backend is being hit in some way, whether by reading a db or by concatenating flat files or something else? Or are they just using a local cache that isn't pushed out to CloudFlare because there would be so much churn that pushing to CF wouldn't be a benefit?
- MichaelGG 11y ago
  Are they using Railgun to only transmit changes? Railgun seems like the neatest thing CF offers, as, AFAIK, you get the bandwidth benefits of caching without having to deal with invalidation.
Alex3917 11y ago
Not sure, but cloudfare supposedly speeds up content delivery even for dynamic content where you need a new db hit each time. The idea is that it leaves open a persistent connection with the webserver, which means A) less work creating and destroying connections B) each new connection isn't starting off as throttled.
- logicuce 11y ago
  They have tech to do even better. See here https://www.cloudflare.com/railgun https://www.cloudflare.com/railgun That is how you can make it faster even when somebody is logged in and you are delivering a somewhat personalized experience.