3 ms·

The root cause (1) is the data store should not have been available on the underlay network. Anything connected to an underlay network is a ticking time bomb.

• jodrellblank 2y ago
  It seems from the article that AT&T uploaded data to a cloud service, protected by username and password, and someone obtained credentials or breached the cloud service. What does that have to do with 'underlay networks' and wow is that "the root cause of 99% of breaches"?
  • gz5 2y ago
    An attacker who gets username/pw still can't get on the overlay network (the overlay requires credentials which can't easily be stolen or compromised, e.g. a private key signed X.509 certificate). Yes, because 99% of attacks use the underlay network to access the target and exfiltrate the data. Said the other way, an attacker didn't physically walk into a Snowflake data center, console into the right server, and walk out with all the data.
    • Aloisius 2y ago
      That sounds more like the lack of certificate-based authentication (or some other stronger authentication method) was the problem, not the lack of a private overlay network. After all, plenty of private overlay networks use simple username/password auth or no auth at all.
      • gz5 2y ago
        Agree, good point, the overlay needs to do strong identity, authN, authZ. The critical part the overlay adds to traditional auth is making the server unreachable from the underlay networks, reducing attack surface by billions. Meaning: + Let's say the server did have good auth, but there was a bug, misconfig, zero day, etc. (one of the myriads of proximate causes). + Since the server is available on the underlay network, that vulnerability can be exploited by anyone on the underlay (billions Internet nodes). + In contrast, making the server only available on the overlay, reduces the attack surface from billions of Internet nodes to the nodes which can ID, authN and authZ (for that particular server) on the overlay.
  • jvanderbot 2y ago
    I doubt they "breeched the cloud service" provider. They almost certainly exploited no 2fa controls on the clients access via the clients network, which is what GP was saying. If you're on a businesses network it's too easy to get at their cloud storage or dbs because they should be on a secure overlay network.
  • wmf 2y ago
    OP is using weird terminology. It would probably be clearer to say "Anything connected to the Internet is a ticking time bomb. Any servers or admins which need to talk to the database should instead use a VPN." which indeed was best practice until recently.
    • mbreese 2y ago
      > indeed was best practice until recently But we should remember why it's not always considered best practices... you shouldn't assume that your private network is any more secure than the public network. When you have too many devices attached to that private (overlay?) network, it can be at just as much risk as if it was on the public internet. So, the zero-trust model is that you don't trust anything... public... private... it should all be untrusted. Given that this was a "third-party cloud provider", I'm assuming that it was a credential leak and they only have username/password protections. Moreover, I doubt you'd have been able to add the provider's DB to an ATT based private VPN/network.
      • gz5 2y ago
        yep was trying to avoid word which carry varying connotations, e.g. vpn or zero trust. zero implicit trust is likely the best term? you have to trust something, but enforce (and therefore trust) strong (not network based) identity, authN and authZ. this can be done anywhere via a software-only overlay. a litmus test is server iptables (to use an example) looks like: iptables -P INPUT DROP iptables -P FORWARD DROP and the only route outbound from the server is to the private overlay on one port, and that server still can't make those connections unless it is strongly identified and authenticated, and the overlay will not connect the client and server unless they are both authorized to communicate for that particular service(1) (1)so for example if there is a zero day causing the 'server' to try to communicate with some_IP then the private overlay will not accept the connection, even though it is coming from the server
        • mbreese 2y ago
          For highly secured services, I completely see the rationale for a private overlayed network. Tailscale, et al are great for this, where you're only exposing services to members of the private network. The problems start when people make the assumption that the private network is a secured network. I don't think any of this would have mattered to ATT, as the breach was from a third party that wouldn't have been on a private network anyway. But, that would be a great service bonus -- only being able to connect to a service via a user-configurable private overlay network. It would be nice, but highly impractical... I can't even begin thinking about how customer support would be able to handle a scheme like this.
• biggc 2y ago
  What? Has anyone published an RCA that confirms this? Is this how the data was ex filtrated from Snowflake? Or did ATT’s Snowflake credentials leak?
• reaperducer 2y ago
  Software, not private circuits If only AT&T had some kind of way for its computers to talk to one another without going over the public internet…
• deleted 2y ago
  [deleted]