5 ms·

Any info about your devs doing only client side verification of the beta flag? Seems like a pretty bad practice.

Operyl 4y ago
Usually rollouts like that are slow to prevent a flood of people using it at once, and to slowly open the flood gates. Not many people will manually change the flag, either. I don't think the fact that the feature flag was just client side is a "smoking gun".
- brightball 4y ago
  Agreed. Feature flags can be complicated to implement both client side and server side. Ideally you want both but it doesn't always work out that way.
ejcx 4y ago
What was missing was the server side ownership check. We decide which customer owns the real "example.com" which is very battle tested logic, but had missed the check in this new service. The client side validation is expected too, though
- Operyl 4y ago
  I'm curious, if you'll disclose it, what the exclusionary policy was. Enterprise zones?
- dreadlordbone 4y ago
  Tracking, thank you.
OJFord 4y ago
It's obviously not good in general, it's 'obscurity' (as in 'is not security') really, but it seems pretty harmless for a gradual roll-out feature toggle? If someone cares enough and knows enough to find they can get past it, let them play with the beta feature? After all, it got them this responsible disclosure. It wasn't part of the vulnerability, it just allowed OP (who happened not to be legitimately in the beta) to find it.
- BeefWellington 4y ago
  > It's obviously not good in general, it's 'obscurity' (as in 'is not security') really, but it seems pretty harmless for a gradual roll-out feature toggle? Not at all. How often do people complain of temporary solutions becoming permanent? Doing it wrong out of the gate is a surefire way to ensure it makes it to production if there's no further review.
  - OJFord 4y ago
    I don't understand your comment at all. I'm not saying anything is 'ok for now while it's a beta'. I'm saying 'client-side validation of beta feature toggles is pretty much fine'. The beta feature had a very bad bug that allowed hijacking other people's email. That is entirely independent of controlling access to the beta feature. It's only mentioned in the write-up because OP wouldn't have been able to explore bugs in the beta feature without finding access to it first, which he didn't otherwise have.
- systemvoltage 4y ago
  Well, the whole point of Beta program is to limit the participation and thus the exposure of attack vectors, and reduce the impact during the testing phase.
  - OJFord 4y ago
    I suppose I wouldn't personally think of it as limiting exposure like that - prod is prod, beta feature or not - but if one does for a given product then yes sure it's a bad bug. Do we really think Cloudflare Email Routing private beta was private to somehow trusted parties only though? Presumably 'N-mutual trusted parties' too, for regulatory compliance. I assume not; not least because the product security lead is here in the comments saying they vetted logs etc. after the fact to ensure that only OP took advantage of this.
    - shyn3 4y ago
      I wouldn't rely on any companies logs or audit trail.

Usually rollouts like that are slow to prevent a flood of people using it at once, and to slowly open the flood gates. Not many people will manually change the flag, either. I don't think the fact that the feature flag was just client side is a "smoking gun".

brightball 4y ago
Agreed. Feature flags can be complicated to implement both client side and server side. Ideally you want both but it doesn't always work out that way.

ejcx 4y ago

What was missing was the server side ownership check. We decide which customer owns the real "example.com" which is very battle tested logic, but had missed the check in this new service. The client side validation is expected too, though

Operyl 4y ago
I'm curious, if you'll disclose it, what the exclusionary policy was. Enterprise zones?
dreadlordbone 4y ago
Tracking, thank you.

OJFord 4y ago

It's obviously not good in general, it's 'obscurity' (as in 'is not security') really, but it seems pretty harmless for a gradual roll-out feature toggle? If someone cares enough and knows enough to find they can get past it, let them play with the beta feature? After all, it got them this responsible disclosure. It wasn't part of the vulnerability, it just allowed OP (who happened not to be legitimately in the beta) to find it.

BeefWellington 4y ago
> It's obviously not good in general, it's 'obscurity' (as in 'is not security') really, but it seems pretty harmless for a gradual roll-out feature toggle? Not at all. How often do people complain of temporary solutions becoming permanent? Doing it wrong out of the gate is a surefire way to ensure it makes it to production if there's no further review.
- OJFord 4y ago
  I don't understand your comment at all. I'm not saying anything is 'ok for now while it's a beta'. I'm saying 'client-side validation of beta feature toggles is pretty much fine'. The beta feature had a very bad bug that allowed hijacking other people's email. That is entirely independent of controlling access to the beta feature. It's only mentioned in the write-up because OP wouldn't have been able to explore bugs in the beta feature without finding access to it first, which he didn't otherwise have.
systemvoltage 4y ago
Well, the whole point of Beta program is to limit the participation and thus the exposure of attack vectors, and reduce the impact during the testing phase.
- OJFord 4y ago
  I suppose I wouldn't personally think of it as limiting exposure like that - prod is prod, beta feature or not - but if one does for a given product then yes sure it's a bad bug. Do we really think Cloudflare Email Routing private beta was private to somehow trusted parties only though? Presumably 'N-mutual trusted parties' too, for regulatory compliance. I assume not; not least because the product security lead is here in the comments saying they vetted logs etc. after the fact to ensure that only OP took advantage of this.
  - shyn3 4y ago
    I wouldn't rely on any companies logs or audit trail.