4 ms·

You should use private endpoints and private link services and turn-off public access to every resource except components that need public Internet access. Sure

You should use private endpoints and private link services and turn-off public access to every resource except components that need public Internet access. Sure it costs some $ but why would a fortune 500 expose their DB endpoint publicly?

onionisafruit 5y ago
> why would a fortune 500 expose their DB endpoint publicly? Incompetence. It’s not the exclusive domain of small companies. I’ve dealt with some very incompetent people at Fortune 500 companies. As a matter of fact, the very people responsible for the mishap we are discussing now work at a Fortune 500 company.
- gautamdivgi 5y ago
  I do apologize, I didn't mean to imply incompetence at small companies. I can understand smaller companies not using private endpoints/private link services for cost reasons. I can't fathom why fortune 500's would not use them to secure their resources.
  - onionisafruit 5y ago
    No apology needed. I’m the one who brought up small companies. I just wanted to point out that big businesses don’t necessarily know what they are doing. People who don’t work for one assume they are well oiled machines, and people who do work for a big business assume every other big business is a well oiled machine.
    - effingwewt 5y ago
      Especially these days as OPSec is seen more as a cost-center than a lawsuit buffer. For some reason they will keep millions on the books specifically for lawsuits, but shelling out $ for proper security is unfathomable (see- T-mobile).
    - dhosek 5y ago
      I was in on discussions for a former employer migrating some of their services and data to Azure. Since it was a health insurance company, there was a great deal of care taken about securing the data. I think a big part of that was that a few years back, there was a well-publicized data breach that impacted the company and few things can generate caution better than a really expensive mistake.
- onion2k 5y ago
  I really hate "incompetence" as an explanation for anything, but especially something as complex as software and security in particular. Obviously, it's easy to blame a single person as incompetent, but the reality is that their work was certainly part of a team effort. Are all those people incompetent? They were using Azure, which presumably didn't warn them well enough, so are the Azure team incompetent for deploying a dangerous application? Clearly the technical authors who wrote and reviewed and published the Azure docs did a bad job of explaining CosmosDB's security model. If these Fortune500 companies employed a security team to test and audit their cloud apps then they missed the problem too. Or maybe they didn't, but some manager or product owner didn't read the report properly, so that security team were incompetent for failing to report the problem well. This is how everything works in pretty much every industry. There are layers upon layers of complexity in everything, and no one has enough oversight to take full responsibility for some mistake that occurred somewhere down the rabbit hole. Somewhere, somehow you could probably call someone's mistake incompetence, but doing that relieves everyone else of their little part in developing a chain of tools and applications that enabled that incompetence. If we heap all the blame on a single individual then no one else has any reason to improve it. We can just say "OMG, Chris was terrible! I'm so glad he's been fired now everything will be perfect!" until the whole sorry mess happens again. Instead if we accept that mistakes are inevitable, and we accept that anyone can make one, then we're driven to build systems and processes that include guards against mistakes. Applications that check and validate things automatically, even if it's hard and expensive. That's how you get to robust software that doesn't fail like the thing in the article. Blaming individuals will never get you to that point.
  - howaboutnope 5y ago
    > Applications that check and validate things automatically, even if it's hard and expensive. Then you still need to be competent enough to to assess that this is the case. You can't judge code doing something correctly automatically if you don't even know what that thing is and can't do it yourself.
  - scns 5y ago
    My guess would be, based on The Gervais Principle and on my predjudice that dissenting opinions aren't viewed favourably and may be detrimental <edit>to your</edit> career, that we have a "The emperors' new clothes" situation here. The people authorized to make decisions may be clueless (GP) and the people pointing out possible problems have been gotten rid of. There is a turkish proverb: "The fish starts stinking at the head". The Peter Principle from 1969! might apply too. https://en.wikipedia.org/wiki/Peter_principle https://en.wikipedia.org/wiki/Peter_principle
- w3ll_w3ll_w3ll 5y ago
  It's not just incompetence. Apart from costs, there are some strange limitation and very specific feature missing if you use Azure PaaS DBs with private endpoint enabled. It's like private access has been added as an afterthought and lags behind the "normal" (public) access.
EE84M3i 5y ago
The article doesn't make it clear, but likely it would still be effected. This is really a container breakout issue between customer's jupyter notebooks.
jnorthrop 5y ago
I agree with you but you can't just throw around the phrase "sure it costs some $" as if that isn't a driving force in everything a big company does. My company is going hard into Azure and I'm making the same recommendations as you suggest, but it is a hard sell precisely because of the cost.

onionisafruit 5y ago

> why would a fortune 500 expose their DB endpoint publicly? Incompetence. It’s not the exclusive domain of small companies. I’ve dealt with some very incompetent people at Fortune 500 companies. As a matter of fact, the very people responsible for the mishap we are discussing now work at a Fortune 500 company.

gautamdivgi 5y ago
I do apologize, I didn't mean to imply incompetence at small companies. I can understand smaller companies not using private endpoints/private link services for cost reasons. I can't fathom why fortune 500's would not use them to secure their resources.
- onionisafruit 5y ago
  No apology needed. I’m the one who brought up small companies. I just wanted to point out that big businesses don’t necessarily know what they are doing. People who don’t work for one assume they are well oiled machines, and people who do work for a big business assume every other big business is a well oiled machine.
  - effingwewt 5y ago
    Especially these days as OPSec is seen more as a cost-center than a lawsuit buffer. For some reason they will keep millions on the books specifically for lawsuits, but shelling out $ for proper security is unfathomable (see- T-mobile).
  - dhosek 5y ago
    I was in on discussions for a former employer migrating some of their services and data to Azure. Since it was a health insurance company, there was a great deal of care taken about securing the data. I think a big part of that was that a few years back, there was a well-publicized data breach that impacted the company and few things can generate caution better than a really expensive mistake.
onion2k 5y ago
I really hate "incompetence" as an explanation for anything, but especially something as complex as software and security in particular. Obviously, it's easy to blame a single person as incompetent, but the reality is that their work was certainly part of a team effort. Are all those people incompetent? They were using Azure, which presumably didn't warn them well enough, so are the Azure team incompetent for deploying a dangerous application? Clearly the technical authors who wrote and reviewed and published the Azure docs did a bad job of explaining CosmosDB's security model. If these Fortune500 companies employed a security team to test and audit their cloud apps then they missed the problem too. Or maybe they didn't, but some manager or product owner didn't read the report properly, so that security team were incompetent for failing to report the problem well. This is how everything works in pretty much every industry. There are layers upon layers of complexity in everything, and no one has enough oversight to take full responsibility for some mistake that occurred somewhere down the rabbit hole. Somewhere, somehow you could probably call someone's mistake incompetence, but doing that relieves everyone else of their little part in developing a chain of tools and applications that enabled that incompetence. If we heap all the blame on a single individual then no one else has any reason to improve it. We can just say "OMG, Chris was terrible! I'm so glad he's been fired now everything will be perfect!" until the whole sorry mess happens again. Instead if we accept that mistakes are inevitable, and we accept that anyone can make one, then we're driven to build systems and processes that include guards against mistakes. Applications that check and validate things automatically, even if it's hard and expensive. That's how you get to robust software that doesn't fail like the thing in the article. Blaming individuals will never get you to that point.
- howaboutnope 5y ago
  > Applications that check and validate things automatically, even if it's hard and expensive. Then you still need to be competent enough to to assess that this is the case. You can't judge code doing something correctly automatically if you don't even know what that thing is and can't do it yourself.
- scns 5y ago
  My guess would be, based on The Gervais Principle and on my predjudice that dissenting opinions aren't viewed favourably and may be detrimental <edit>to your</edit> career, that we have a "The emperors' new clothes" situation here. The people authorized to make decisions may be clueless (GP) and the people pointing out possible problems have been gotten rid of. There is a turkish proverb: "The fish starts stinking at the head". The Peter Principle from 1969! might apply too. https://en.wikipedia.org/wiki/Peter_principle https://en.wikipedia.org/wiki/Peter_principle
w3ll_w3ll_w3ll 5y ago
It's not just incompetence. Apart from costs, there are some strange limitation and very specific feature missing if you use Azure PaaS DBs with private endpoint enabled. It's like private access has been added as an afterthought and lags behind the "normal" (public) access.

EE84M3i 5y ago

The article doesn't make it clear, but likely it would still be effected. This is really a container breakout issue between customer's jupyter notebooks.

jnorthrop 5y ago

I agree with you but you can't just throw around the phrase "sure it costs some $" as if that isn't a driving force in everything a big company does. My company is going hard into Azure and I'm making the same recommendations as you suggest, but it is a hard sell precisely because of the cost.