6 ms·

How We Extended CloudFlare into Mainland China

60 points by eastdakota 11y ago

aianus 11y ago
Chinese Internet infrastructure is so backwards it's kind of funny.
- dang 11y ago
  > China is so backwards it's kind of funny. It breaks the HN guidelines to make slurs against anyone's country here. Please don't do that again.
  - aianus 11y ago
    I'm sorry you took it that way. Replaced China with Chinese Internet infrastructure.
  - deleted 11y ago
    [deleted]
ck2 11y ago
Hey cloudflare, how can you do all these impressive things yet your dns editor ui is so bad? Try exporting your dns and then importing, compare the mess it makes on what should be a simple zone file, TXT records are slashed, records are out of order, TTL is changed, and the columns are so narrow you cannot even inspect the records for errors.
needcaffeine 11y ago
This alone will get me to sign up for CloudFlare enterprise, assuming that getting an ICP license is actually easy to do. What are the advantages of CloudFlare over say...ChinaCache, which promises hundreds of edge locations inside mainland China?
- aembleton 11y ago
  Foreign companies (unless they have a Chinese subsidiary) cannot apply for an ICP. https://support.cloudflare.com/hc/en-us/articles/209714777 https://support.cloudflare.com/hc/en-us/articles/209714777
  - needcaffeine 11y ago
    That's correct. ChinaCache overcomes this by submitting the request using a proxy entity that you pay for every month. It all really comes down to money.
bpolverini 11y ago
One can only imagine what kind of Faustian bargain Cloudflare had to concoct in order to make this all legal. Once Baidu and the PSB have managed to extract enough useful intellectual property from Cloudflare, it will only be a matter of time until they find a reason to block Cloudflare and replace it with a domestic service that can be even more tightly controlled. Cloudflare is phenomenally competent, but Matthew Prince is likely in for a serious surprise that has nothing to do with technological innovation and everything to do with sociology and a different culture.
- joe_the_user 11y ago
  Yes, it's dishonest for them to describe China's censorship regime as "regulation". Like any dictatorial regime, China's censorship is fundamentally "lawless" in the sense that there's no codified statement of what can and can't be said. Rather China maintains a constant threat that saying anything critical of the regime may result in sanctions, often retroactively. And if anyone references the US, the NSA's surveillance regime has the same lawless quality despite the fig leaf of (secret) courts. We can at least be happy the US doesn't have all the overt repressive mechanism of China - yet.
- MichaelGG 11y ago
  The bargain is probably something like: "Customers using our China POPs give us money. We like money." And is it thought that China lacks the resources to infiltrate CloudFlare? The KGB compromised the CIA and FBI over many years for paltry sums (under a million). If China really wanted to extract IP they can certainly do so. While operational security (like HSMs and procedures) might stop people from walking off with key material, I find it really hard to believe companies can stop adversaries from walking off with source code, know-how, technical plans - things that employees need access to.
- carterehsmith 11y ago
  Hmmm. Many people have this perception that Cloudflare sells "protection" to DDOS "victims", while providing a comfy hosting place for DDOS "providers". Good business, selling weapons to two opposed groups.
alphang 11y ago
Is CloudFlare's desire to extend into the lucrative Chinese market going to cause reduce service for Hong Kong's pro-democracy customers? http://www.forbes.com/sites/parmyolson/2014/11/20/the-largest-cyber-attack-in-history-has-been-hitting-hong-kong-sites/ http://www.forbes.com/sites/parmyolson/2014/11/20/the-larges...
song 11y ago
> Baidu's regulatory expertise also helped to solve what previously seemed like an insurmountable problem. They developed a process whereby ICP license applications could be automatically submitted on behalf of CloudFlare customers. This removes the burden of individual customers having to navigate local licensing requirements. Translation: "Baidu's regulatory expertise" means "Baidu's guanxi and relationship with key decision makers in the country that invested and will profit from the company growth". It's always funny how corruption gets renamed to better sounding words like "expertise".
- justicezyx 11y ago
  No it's expertise... You are just too naive to believe that gov relationship is something less important or could be obtained without much effort...
- phn 11y ago
  Why is the relationship with key decision makers considered corruption? I get it's not "expertise", but I don't get why it's corruption either.
  - song 11y ago
    It's corruption because the relationship usually involve the key decision makers having either stock in that company or being rewarded financially in one way or another.
philip1209 11y ago
If Cloudflare is not MITMing SSL in China unless you explicitly allow it, do caches still serve from there? I guess the scenario I am wondering about is whether javascript caches could hypothetically be poisoned by a malicious actor even if SSL traffic is still not technically being MITM'd at these servers.
MichaelGG 11y ago
How is their universal SSL stuff being handled in China? Is there a separate root they use? Since certs are shared (with lots of SANs) if my site is sharing a cert with one that opts-in to China access, does that mean the private key for that cert is now available to China?
- eastdakota 11y ago
  We're using Keyless SSL when HTTPS goes live inside China. That means we can handle HTTPS without any keys being stored inside China.
- skybrian 11y ago
  It seems possible that the censors don't need the private keys because they somehow get access to symmetric keys used to terminate the connection. That would be bad, but at least doesn't affect users outside China. (To be clear, this is pure speculation; I have no actual knowledge of such things.)
micah_chatt 11y ago
I've never paid much attention CloudFlare's datacenter map before, but I'm quite surprised to not see anything in India or near Egypt/Israel.
- jgrahamc 11y ago
  With CloudFlare expansion that's not a matter of if but when.
- zhemao 11y ago
  A lot of it has to do with whether local ISPs and utility companies are reliable enough to get satisfactory availability. I once worked at a company that operated a few Indian PoPs and we always had trouble with them. Sometimes we would simply lose all connectivity to them.
pakled_engineer 11y ago
>Although we may not be able to announce certain content from within China, or any other country in which certain content may be prohibited, we continue to serve it across the Internet from the rest of the network. Does this mean that the CPC can now have 2 factor censorship? If they get past the great firewall they now have to get past Cloudflare too "We're sorry, this content is not available in your country as per request of Ministry of Public Security".
microsage 11y ago
Interestingly, the linked article isn't actually available inside of mainland China at the moment. Maybe they're not running their own blog through the China-enabled portion of their CDN?
- tellarin 11y ago
  Works fine for me since the story was posted. In Beijing here.
a_c 11y ago
> After a survey of our customer base, we determined that more than 99% of our customers’ websites are locally available in China today. Since they censor most foreign website, I bet average chinese users doesn't even aware of the existence of overseas website
rurban 11y ago
I kind of find it funny that CloudFlare who occasionally protects their customers from getting DDOS'ed by probable Chinese state actors, now gets behind the curtain and will now have to find new ways to protect their customers getting DDOS'ed from inside if they publish content which the Chinese authorities don't like. I guess it's easier now for them to take it down without resorting to warfare. But then again, this content will not be permitted inside. So it will stay the same. Maybe it will change now from ICMP over clever javascript bombs to DNS attacks. Which is how CloudFlare protects their domains.

60 points by eastdakota 11y ago

aianus 11y ago

Chinese Internet infrastructure is so backwards it's kind of funny.

dang 11y ago
> China is so backwards it's kind of funny. It breaks the HN guidelines to make slurs against anyone's country here. Please don't do that again.
- aianus 11y ago
  I'm sorry you took it that way. Replaced China with Chinese Internet infrastructure.
- deleted 11y ago
  [deleted]

ck2 11y ago

Hey cloudflare, how can you do all these impressive things yet your dns editor ui is so bad? Try exporting your dns and then importing, compare the mess it makes on what should be a simple zone file, TXT records are slashed, records are out of order, TTL is changed, and the columns are so narrow you cannot even inspect the records for errors.

needcaffeine 11y ago

This alone will get me to sign up for CloudFlare enterprise, assuming that getting an ICP license is actually easy to do. What are the advantages of CloudFlare over say...ChinaCache, which promises hundreds of edge locations inside mainland China?

aembleton 11y ago
Foreign companies (unless they have a Chinese subsidiary) cannot apply for an ICP. https://support.cloudflare.com/hc/en-us/articles/209714777 https://support.cloudflare.com/hc/en-us/articles/209714777
- needcaffeine 11y ago
  That's correct. ChinaCache overcomes this by submitting the request using a proxy entity that you pay for every month. It all really comes down to money.

bpolverini 11y ago

One can only imagine what kind of Faustian bargain Cloudflare had to concoct in order to make this all legal. Once Baidu and the PSB have managed to extract enough useful intellectual property from Cloudflare, it will only be a matter of time until they find a reason to block Cloudflare and replace it with a domestic service that can be even more tightly controlled. Cloudflare is phenomenally competent, but Matthew Prince is likely in for a serious surprise that has nothing to do with technological innovation and everything to do with sociology and a different culture.

joe_the_user 11y ago
Yes, it's dishonest for them to describe China's censorship regime as "regulation". Like any dictatorial regime, China's censorship is fundamentally "lawless" in the sense that there's no codified statement of what can and can't be said. Rather China maintains a constant threat that saying anything critical of the regime may result in sanctions, often retroactively. And if anyone references the US, the NSA's surveillance regime has the same lawless quality despite the fig leaf of (secret) courts. We can at least be happy the US doesn't have all the overt repressive mechanism of China - yet.
MichaelGG 11y ago
The bargain is probably something like: "Customers using our China POPs give us money. We like money." And is it thought that China lacks the resources to infiltrate CloudFlare? The KGB compromised the CIA and FBI over many years for paltry sums (under a million). If China really wanted to extract IP they can certainly do so. While operational security (like HSMs and procedures) might stop people from walking off with key material, I find it really hard to believe companies can stop adversaries from walking off with source code, know-how, technical plans - things that employees need access to.
carterehsmith 11y ago
Hmmm. Many people have this perception that Cloudflare sells "protection" to DDOS "victims", while providing a comfy hosting place for DDOS "providers". Good business, selling weapons to two opposed groups.

alphang 11y ago

Is CloudFlare's desire to extend into the lucrative Chinese market going to cause reduce service for Hong Kong's pro-democracy customers? http://www.forbes.com/sites/parmyolson/2014/11/20/the-largest-cyber-attack-in-history-has-been-hitting-hong-kong-sites/ http://www.forbes.com/sites/parmyolson/2014/11/20/the-larges...

song 11y ago

> Baidu's regulatory expertise also helped to solve what previously seemed like an insurmountable problem. They developed a process whereby ICP license applications could be automatically submitted on behalf of CloudFlare customers. This removes the burden of individual customers having to navigate local licensing requirements. Translation: "Baidu's regulatory expertise" means "Baidu's guanxi and relationship with key decision makers in the country that invested and will profit from the company growth". It's always funny how corruption gets renamed to better sounding words like "expertise".

justicezyx 11y ago
No it's expertise... You are just too naive to believe that gov relationship is something less important or could be obtained without much effort...
phn 11y ago
Why is the relationship with key decision makers considered corruption? I get it's not "expertise", but I don't get why it's corruption either.
- song 11y ago
  It's corruption because the relationship usually involve the key decision makers having either stock in that company or being rewarded financially in one way or another.

philip1209 11y ago

If Cloudflare is not MITMing SSL in China unless you explicitly allow it, do caches still serve from there? I guess the scenario I am wondering about is whether javascript caches could hypothetically be poisoned by a malicious actor even if SSL traffic is still not technically being MITM'd at these servers.

MichaelGG 11y ago

How is their universal SSL stuff being handled in China? Is there a separate root they use? Since certs are shared (with lots of SANs) if my site is sharing a cert with one that opts-in to China access, does that mean the private key for that cert is now available to China?

eastdakota 11y ago
We're using Keyless SSL when HTTPS goes live inside China. That means we can handle HTTPS without any keys being stored inside China.
skybrian 11y ago
It seems possible that the censors don't need the private keys because they somehow get access to symmetric keys used to terminate the connection. That would be bad, but at least doesn't affect users outside China. (To be clear, this is pure speculation; I have no actual knowledge of such things.)

micah_chatt 11y ago

I've never paid much attention CloudFlare's datacenter map before, but I'm quite surprised to not see anything in India or near Egypt/Israel.

jgrahamc 11y ago
With CloudFlare expansion that's not a matter of if but when.
zhemao 11y ago
A lot of it has to do with whether local ISPs and utility companies are reliable enough to get satisfactory availability. I once worked at a company that operated a few Indian PoPs and we always had trouble with them. Sometimes we would simply lose all connectivity to them.

pakled_engineer 11y ago

>Although we may not be able to announce certain content from within China, or any other country in which certain content may be prohibited, we continue to serve it across the Internet from the rest of the network. Does this mean that the CPC can now have 2 factor censorship? If they get past the great firewall they now have to get past Cloudflare too "We're sorry, this content is not available in your country as per request of Ministry of Public Security".

microsage 11y ago

Interestingly, the linked article isn't actually available inside of mainland China at the moment. Maybe they're not running their own blog through the China-enabled portion of their CDN?

tellarin 11y ago
Works fine for me since the story was posted. In Beijing here.

a_c 11y ago

> After a survey of our customer base, we determined that more than 99% of our customers’ websites are locally available in China today. Since they censor most foreign website, I bet average chinese users doesn't even aware of the existence of overseas website

rurban 11y ago

I kind of find it funny that CloudFlare who occasionally protects their customers from getting DDOS'ed by probable Chinese state actors, now gets behind the curtain and will now have to find new ways to protect their customers getting DDOS'ed from inside if they publish content which the Chinese authorities don't like. I guess it's easier now for them to take it down without resorting to warfare. But then again, this content will not be permitted inside. So it will stay the same. Maybe it will change now from ICMP over clever javascript bombs to DNS attacks. Which is how CloudFlare protects their domains.