4 ms·

> It's not posted on SHD as the issue has impacted our ability to post there. Is that not a massive catch-22 for a service dashboard?

by _0o6v 6y ago

snazz 6y ago
This has happened a few times before, actually. Dogfooding is good, but not for status pages! Cloudflare does it right for their status page (https://www.cloudflarestatus.com https://www.cloudflarestatus.com). They don't use Cloudflare itself for it (you can tell because /cdn-cgi/trace returns nothing), the actual backend is Atlassian Statuspage, their TLS certificate is issued by Let's Encrypt instead of Cloudflare itself, and it's on a completely separate domain for DNS purposes.
- toong 6y ago
  Have you checked https://www.githubstatus.com/ https://www.githubstatus.com/ ? ;-)
  - lights0123 6y ago
    GitHub doesn't own their own datacenters.
- judge2020 6y ago
  They do use their own registrar though: $ whois cloudflarestatus.com Registrar: Cloudflare, Inc.
freehunter 6y ago
Reminds me of a recent outage from IBM Cloud, where the VPN was hosted on IBM Cloud so employees couldn’t log in to fix it, and the email is hosted on IBM Cloud so support teams couldn’t email customers to let them know and even access to their Twitter was behind the non-functional VPN so they couldn’t tweet during the outage either.
- kamyarg 6y ago
  This sounds like a true nightmare. Do you have a link for more details?
  - freehunter 6y ago
    I actually heard the details on a cloud-related podcast, but I found a transcript of the episode here: https://www.lastweekinaws.com/podcast/aws-morning-brief/whiteboard-confessional-the-day-ibm-cloud-dissipated/ https://www.lastweekinaws.com/podcast/aws-morning-brief/whit... The relevant bit: >[customers] were texting with their account managers, because the account managers had no access to any internal systems. Reportedly, the corporate VPN was not working. My thesis is... everything was single-tracking through a corporate VPN that itself was subject to this disruption... their traditional tweets have been done through an enterprise social media client called Sprinklr

> It's not posted on SHD as the issue has impacted our ability to post there. Is that not a massive catch-22 for a service dashboard?

by _0o6v 6y ago

> It's not posted on SHD as the issue has impacted our ability to post there. Is that not a massive catch-22 for a service dashboard?

snazz 6y ago

This has happened a few times before, actually. Dogfooding is good, but not for status pages! Cloudflare does it right for their status page (https://www.cloudflarestatus.com https://www.cloudflarestatus.com). They don't use Cloudflare itself for it (you can tell because /cdn-cgi/trace returns nothing), the actual backend is Atlassian Statuspage, their TLS certificate is issued by Let's Encrypt instead of Cloudflare itself, and it's on a completely separate domain for DNS purposes.

toong 6y ago
Have you checked https://www.githubstatus.com/ https://www.githubstatus.com/ ? ;-)
- lights0123 6y ago
  GitHub doesn't own their own datacenters.
judge2020 6y ago
They do use their own registrar though: $ whois cloudflarestatus.com Registrar: Cloudflare, Inc.

freehunter 6y ago

Reminds me of a recent outage from IBM Cloud, where the VPN was hosted on IBM Cloud so employees couldn’t log in to fix it, and the email is hosted on IBM Cloud so support teams couldn’t email customers to let them know and even access to their Twitter was behind the non-functional VPN so they couldn’t tweet during the outage either.

kamyarg 6y ago
This sounds like a true nightmare. Do you have a link for more details?
- freehunter 6y ago
  I actually heard the details on a cloud-related podcast, but I found a transcript of the episode here: https://www.lastweekinaws.com/podcast/aws-morning-brief/whiteboard-confessional-the-day-ibm-cloud-dissipated/ https://www.lastweekinaws.com/podcast/aws-morning-brief/whit... The relevant bit: >[customers] were texting with their account managers, because the account managers had no access to any internal systems. Reportedly, the corporate VPN was not working. My thesis is... everything was single-tracking through a corporate VPN that itself was subject to this disruption... their traditional tweets have been done through an enterprise social media client called Sprinklr