9 ms·

Cloudflare Experiencing Latency Issues

151 points by z0a 5y ago

spurgu 5y ago
Yeah seems to apply to all the sites I've tried so far.
tailspin2019 5y ago
All my sites running through Cloudflare Tunnel are very very slow, but still just about online. As a side note, I'll take this opportunity call out the superb Checkmk monitoring system which alerted me to this. I don't see Checkmk mentioned on HN that often... https://checkmk.com https://checkmk.com EDIT: Seems to be fixed. Good job!
- donmcronald 5y ago
  I don’t think I’d use it even if they have a self-hosted version. The way they set their pricing based on “number of services” seems like the kind of tactic you use when you want to intentionally make things confusing so you can extract as much value as possible. What happened to honest businesses with fair, easy to understand pricing?
  - tailspin2019 5y ago
    > What happened to honest businesses with fair, easy to understand pricing? Well in my case the pricing is very easy to understand. It's free! I only have < 25 hosts so I self-host the open source version on a $5/month DigitalOcean instance (ironically also reverse proxied through Cloudflare) So I certainly don't think that's exactly dishonest or unfair. It's been rock solid since I've used it. I don't know how many services you'd need to monitor but the starting prices for Standard and Enterprise seem pretty reasonable to me? It probably doesn't scale to a very large operation - but then it's not really "cloud first" monitoring akin to something like Prometheus, so perhaps their target audience isn't really likely to have a huge number of services to monitor.
- marcolussetti 5y ago
  CheckMK offers pretty solid monitoring our of the box, but I think it falls quite short of the mark when you want to add more than the default monitoring. It's still probably the easiest solution for monitoring a bunch of basic VMs, servers, etc. You can set it up and get a solid idea of how it works in only a few hours.
Jamie9912 5y ago
Seems to be a regional thing because i'm not experiencing any issues with Cloudflare hosted things, reaching the SYD PoP
lend000 5y ago
This impacted just about every API I regularly request for ~20 minutes, but it seems to be fixed now. Hopefully permanently.
yRetsyM 5y ago
I'm impressed they've updated their status page so quickly, unlike some of their cloud competitors
- jgrahamc 5y ago
  It's part of our process. Here's the internal timeline T+0 Automatic comms thread created T+1 XXX Is this a P0, do we need a status page? @YYY T+1 YYY Eyes on T+4 ZZZ Yes let's get super-generic status page up @XXX / @YYY - you have one handy? I see it now thx
  - ethbr0 5y ago
    It's definitely a differentiator. cough
jgrahamc 5y ago
Should be cleared up now. Sorry about that.
- tentacleuno 5y ago
  I really like it when people in your position at such a big company post on here, even if it is a brief comment like this. Thank you!
  - jgrahamc 5y ago
    I would have posted faster but I was too busy doing my own little bit of debugging which consisted of 1. dig @1.1.1.1 jgc.org 2. nc -v 104.22.11.223 80 3. curl -v https://jgc.org/cdn-cgi/trace 4. curl -v https://jgc.org/ Hmm #1 was fast so network is routing OK. Hmm #2 was fast so TCP is OK. Hmm #3 was fast so I know (because I worked on that code) that this code path is good. Hmm #4 is slow so that means component X is slow but still working. Of course, in parallel I'm in a conference call with about 40 other people who have actual access to monitoring and systems and other things who can see exactly where things are. But I was damn close with four commands and gave me confidence in what people were saying. But, I have to say, Cloudflare's internal distributed tracing system is pretty cool because I got sent a trace and you could see right where the slowdown was.
    - tailspin2019 5y ago
      Now you're just showing off :)
      - jgrahamc 5y ago
        Allow an old man the fantasy that he still knows how the whole of the system works.
        jatone 5y ago
        sounds like you do. ;)
zertop 5y ago
> Monitoring > Cloudflare has implemented a fix for this issue and is currently monitoring the results. > Posted 1 minute ago. Dec 16, 2021 - 20:44 UTC Fast feedback, communication and fix. Always impressed with them...
chizhik-pyzhik 5y ago
CircleCI still very slow/returning 504s, not sure if related?
deleted 5y ago
[deleted]
jsiaajdsdaa 5y ago
Better than being down down!
amar0c 5y ago
Is this Telia (1299) related maybe ?
mrcnkoba 5y ago
I would love if their customer service was so fast. They keep ghosting us for 7 days. Poking the on a live chat results in "hey we'll look at this". Truly loving the service but we had to "unproxy" our website. When it works, it brings so much value. I'm guessing our issue isn't trivial to solve though.
- jgrahamc 5y ago
  Email me (jgc) the ticket number.
gildedage77 5y ago
Maybe because they're powered by Clickhouse?

151 points by z0a 5y ago

Yeah seems to apply to all the sites I've tried so far.

tailspin2019 5y ago

All my sites running through Cloudflare Tunnel are very very slow, but still just about online. As a side note, I'll take this opportunity call out the superb Checkmk monitoring system which alerted me to this. I don't see Checkmk mentioned on HN that often... https://checkmk.com https://checkmk.com EDIT: Seems to be fixed. Good job!

donmcronald 5y ago
I don’t think I’d use it even if they have a self-hosted version. The way they set their pricing based on “number of services” seems like the kind of tactic you use when you want to intentionally make things confusing so you can extract as much value as possible. What happened to honest businesses with fair, easy to understand pricing?
- tailspin2019 5y ago
  > What happened to honest businesses with fair, easy to understand pricing? Well in my case the pricing is very easy to understand. It's free! I only have < 25 hosts so I self-host the open source version on a $5/month DigitalOcean instance (ironically also reverse proxied through Cloudflare) So I certainly don't think that's exactly dishonest or unfair. It's been rock solid since I've used it. I don't know how many services you'd need to monitor but the starting prices for Standard and Enterprise seem pretty reasonable to me? It probably doesn't scale to a very large operation - but then it's not really "cloud first" monitoring akin to something like Prometheus, so perhaps their target audience isn't really likely to have a huge number of services to monitor.
marcolussetti 5y ago
CheckMK offers pretty solid monitoring our of the box, but I think it falls quite short of the mark when you want to add more than the default monitoring. It's still probably the easiest solution for monitoring a bunch of basic VMs, servers, etc. You can set it up and get a solid idea of how it works in only a few hours.

Jamie9912 5y ago

Seems to be a regional thing because i'm not experiencing any issues with Cloudflare hosted things, reaching the SYD PoP

lend000 5y ago

This impacted just about every API I regularly request for ~20 minutes, but it seems to be fixed now. Hopefully permanently.

yRetsyM 5y ago

I'm impressed they've updated their status page so quickly, unlike some of their cloud competitors

jgrahamc 5y ago
It's part of our process. Here's the internal timeline T+0 Automatic comms thread created T+1 XXX Is this a P0, do we need a status page? @YYY T+1 YYY Eyes on T+4 ZZZ Yes let's get super-generic status page up @XXX / @YYY - you have one handy? I see it now thx
- ethbr0 5y ago
  It's definitely a differentiator. cough

jgrahamc 5y ago

Should be cleared up now. Sorry about that.

tentacleuno 5y ago
I really like it when people in your position at such a big company post on here, even if it is a brief comment like this. Thank you!
- jgrahamc 5y ago
  I would have posted faster but I was too busy doing my own little bit of debugging which consisted of 1. dig @1.1.1.1 jgc.org 2. nc -v 104.22.11.223 80 3. curl -v https://jgc.org/cdn-cgi/trace 4. curl -v https://jgc.org/ Hmm #1 was fast so network is routing OK. Hmm #2 was fast so TCP is OK. Hmm #3 was fast so I know (because I worked on that code) that this code path is good. Hmm #4 is slow so that means component X is slow but still working. Of course, in parallel I'm in a conference call with about 40 other people who have actual access to monitoring and systems and other things who can see exactly where things are. But I was damn close with four commands and gave me confidence in what people were saying. But, I have to say, Cloudflare's internal distributed tracing system is pretty cool because I got sent a trace and you could see right where the slowdown was.
  - tailspin2019 5y ago
    Now you're just showing off :)
    - jgrahamc 5y ago
      Allow an old man the fantasy that he still knows how the whole of the system works.
      - jatone 5y ago
        sounds like you do. ;)

zertop 5y ago

> Monitoring > Cloudflare has implemented a fix for this issue and is currently monitoring the results. > Posted 1 minute ago. Dec 16, 2021 - 20:44 UTC Fast feedback, communication and fix. Always impressed with them...

chizhik-pyzhik 5y ago

CircleCI still very slow/returning 504s, not sure if related?

deleted 5y ago

[deleted]

jsiaajdsdaa 5y ago

Better than being down down!

amar0c 5y ago

Is this Telia (1299) related maybe ?

mrcnkoba 5y ago

I would love if their customer service was so fast. They keep ghosting us for 7 days. Poking the on a live chat results in "hey we'll look at this". Truly loving the service but we had to "unproxy" our website. When it works, it brings so much value. I'm guessing our issue isn't trivial to solve though.

jgrahamc 5y ago
Email me (jgc) the ticket number.

gildedage77 5y ago