4 ms·

Cloudflare is really failing to inspire confidence here. Between them downplaying the issue, refusing to work with the Google engineer who found it, blaming the

by tedivm 10y ago

Cloudflare is really failing to inspire confidence here. Between them downplaying the issue, refusing to work with the Google engineer who found it, blaming the proxy owners (such as google) for not cleaning up their mess quickly enough, and attempting to dump the blame on groups like Ragel- I just have a hard time believing that Cloudflare will ever handle a security incident properly after this mess.

ethbro 10y ago
You mean by being transparent as opposed to some of the other serious security incidents out there? Could you please clarify exactly how they "downplayed the issue"? The impact is an exercise in statistics. They seem to have compiled what logging / statistics they could, then accurately advised everyone of what they found. They turned around an initial fix on the issue in hours, no? As for web caches, what are they supposed to do, other than plead for their owners to delete them ASAP? As for Ragel, not sure how the initial version of the response read, but the one I read clearly pointed out it was a Cloudflare issue. And given the semantics of the issue, I'd cut them some slack if they misidentified the cause the day after they'd been scrambling to fix this. I'd be interested who you do trust to properly handle a security incident if you're looking for better than this. IBM? Microsoft? Apple? Just try and get a 24-hour fix out of them.
- cbr 10y ago
  At least with "blaming the proxy owners (such as google) for not cleaning up their mess quickly enough" I think tedivim is probably referring to: https://news.ycombinator.com/item?id=13721644 https://news.ycombinator.com/item?id=13721644
  - ethbro 10y ago
    I saw that comment when it hit the original thread. I think commenters are conflating Project Zero with Google search cache in the usual version of this sentiment. If I was Cloudflare, and someone had made me aware of this bug, I'd be griping as loudly as possible to get remaining caches cleared ASAP. It's all they really can do. The fact that Bing had unlocated cached documents doesn't impact on Google search being quick or not to clear their own caches. Did Cloudflare cause all this? Yes. Are they working in the interest of their customers to gripe at remaining cachers to refresh their caches? Also yes. Is there something else they could do now that we're here? I can't think of anything more effective.
- tyingq 10y ago
  >not sure how the initial version of the response read, but the one I read clearly pointed out it was a Cloudflare issue Here's a diff of the original and what's there now. https://gist.github.com/anonymous/e8f78f537551465e54a9781d6212a96d https://gist.github.com/anonymous/e8f78f537551465e54a9781d62... To me, it doesn't look like any attempt to change history. They did try to make it more clear that it was their issue, and not Ragel's. But, any initial appearance of that seems accidental.
- tedivm 10y ago
  They downplayed the issue repeatedly. If you read the original Google report Cloudflare did not work with the team all that well. The Google team complained that Cloudflare was not working with them on the message and was downplaying the risk. > https://bugs.chromium.org/p/project-zero/issues/detail?id=1139#c12 https://bugs.chromium.org/p/project-zero/issues/detail?id=11... According to the Google team the draft Cloudflare sent "contains an excellent postmortem, but severely downplays the risk to customers." There was also a ton of conversation here on this site about the fact that they were downplaying the severity. As for web caches- look, it's great that they were proactively working to get things cleaned up. However, despite the fact that Google called in engineers who cancelled their weekend plans to deal with this, and despite the fact that Google is the one who did Cloudflare's job by finding the exploit, this is what Matt Prince had to say- > I agree it's troubling that Google is taking so long. We were working with them to coordinate disclosure after their caches were cleared. While I am thankful to the Project Zero team for their informing us of the issue quickly, I'm troubled that they went ahead with disclosure before Google crawl team could complete the refresh of their own cache. We have continued to escalate this within Google to get the crawl team to prioritize the clearing of their caches as that is the highest priority remaining remediation step. He essentially through the Google team under the bus. It was completely unprofessional. Finally, the Ragel team themselves had to reach out to Cloudflare to get them to change the language on their announcement because it appeared to many people that the Cloudflare discussion was shifting the blame to Ragel. If you really need more information about this just read the OP this thread is part of.

by tedivm 10y ago

ethbro 10y ago

You mean by being transparent as opposed to some of the other serious security incidents out there? Could you please clarify exactly how they "downplayed the issue"? The impact is an exercise in statistics. They seem to have compiled what logging / statistics they could, then accurately advised everyone of what they found. They turned around an initial fix on the issue in hours, no? As for web caches, what are they supposed to do, other than plead for their owners to delete them ASAP? As for Ragel, not sure how the initial version of the response read, but the one I read clearly pointed out it was a Cloudflare issue. And given the semantics of the issue, I'd cut them some slack if they misidentified the cause the day after they'd been scrambling to fix this. I'd be interested who you do trust to properly handle a security incident if you're looking for better than this. IBM? Microsoft? Apple? Just try and get a 24-hour fix out of them.

cbr 10y ago
At least with "blaming the proxy owners (such as google) for not cleaning up their mess quickly enough" I think tedivim is probably referring to: https://news.ycombinator.com/item?id=13721644 https://news.ycombinator.com/item?id=13721644
- ethbro 10y ago
  I saw that comment when it hit the original thread. I think commenters are conflating Project Zero with Google search cache in the usual version of this sentiment. If I was Cloudflare, and someone had made me aware of this bug, I'd be griping as loudly as possible to get remaining caches cleared ASAP. It's all they really can do. The fact that Bing had unlocated cached documents doesn't impact on Google search being quick or not to clear their own caches. Did Cloudflare cause all this? Yes. Are they working in the interest of their customers to gripe at remaining cachers to refresh their caches? Also yes. Is there something else they could do now that we're here? I can't think of anything more effective.
tyingq 10y ago
>not sure how the initial version of the response read, but the one I read clearly pointed out it was a Cloudflare issue Here's a diff of the original and what's there now. https://gist.github.com/anonymous/e8f78f537551465e54a9781d6212a96d https://gist.github.com/anonymous/e8f78f537551465e54a9781d62... To me, it doesn't look like any attempt to change history. They did try to make it more clear that it was their issue, and not Ragel's. But, any initial appearance of that seems accidental.
tedivm 10y ago
They downplayed the issue repeatedly. If you read the original Google report Cloudflare did not work with the team all that well. The Google team complained that Cloudflare was not working with them on the message and was downplaying the risk. > https://bugs.chromium.org/p/project-zero/issues/detail?id=1139#c12 https://bugs.chromium.org/p/project-zero/issues/detail?id=11... According to the Google team the draft Cloudflare sent "contains an excellent postmortem, but severely downplays the risk to customers." There was also a ton of conversation here on this site about the fact that they were downplaying the severity. As for web caches- look, it's great that they were proactively working to get things cleaned up. However, despite the fact that Google called in engineers who cancelled their weekend plans to deal with this, and despite the fact that Google is the one who did Cloudflare's job by finding the exploit, this is what Matt Prince had to say- > I agree it's troubling that Google is taking so long. We were working with them to coordinate disclosure after their caches were cleared. While I am thankful to the Project Zero team for their informing us of the issue quickly, I'm troubled that they went ahead with disclosure before Google crawl team could complete the refresh of their own cache. We have continued to escalate this within Google to get the crawl team to prioritize the clearing of their caches as that is the highest priority remaining remediation step. He essentially through the Google team under the bus. It was completely unprofessional. Finally, the Ragel team themselves had to reach out to Cloudflare to get them to change the language on their announcement because it appeared to many people that the Cloudflare discussion was shifting the blame to Ragel. If you really need more information about this just read the OP this thread is part of.