3 ms·

Hit the nail on the head. As a security person, the marketing and data engineering teams at my company are what keeps me up at night. They inherently desire to

Hit the nail on the head. As a security person, the marketing and data engineering teams at my company are what keeps me up at night. They inherently desire to work with the company’s Crown Jewels of data. They want unfettered access to, and want to give other people access to, an entire data warehouse of some of our company’s most sensitive customer information. And when it comes to securing it properly, they often do not have the same engineering capabilities as software teams.

Kalium 3y ago
As a security person trying to work with marketing and data engineering teams, I've often found them very resistant to learning how to secure things. They generally want to use the tools they know, in the way they know, to do the tasks they understand, on a timeline they find convenient. Anything that tries to get them to understand the risks they are taking or the sensitivity of the data, much less de-risk their workflows, is treated as an obstacle to be routed around. Often, the best I can hope for is a token effort at negotiation where their goal will be to avoid any and all changes on their part. After which I will have to monitor them carefully, because from experience the odds of them backsliding within a week are uncomfortably high. Nothing about this is conducive to producing a healthy environment. When people's idea of "easy" is they can download the company's most sensitive data to their laptop to load into Jupyter, any amount of security controls will come as an imposition.
- bluedemon 3y ago
  A question for you and anyone else in security: Given that many data engineers have a data science, data analytics, BI, or software engineering background, I'm curious if you've noticed any trends in their approach to data security?
  - Kalium 3y ago
    Yes. Generally it can be summarized as "What data security?". Snark aside, there's usually a reflexive assumption that more data is always better and that anything that gets in the way of more data is bad. Anything that limits how data is analyzed is bad. Anything that limits or restricts their choice of tooling or where they use it is rejected. Data scientists and engineers are people who are, often, working with a company's crown jewels. They are trusted with data representing the private lives of hundreds of millions of people assembled in a data warehouse. I want them to have a care. To treat this with due gravitas. All too often, all they seem to see is a neat data set to feed into R on their Macbook.
- wodenokoto 3y ago
  Log in to your vpn, then log in to a website to download a token. Then use said token to log on to a virtual desktop. From here you can open a browser and log in to powerBI, which has access to data in the data warehouse. Yes, such workflows are something to be routed around.
  - Kalium 3y ago
    You're absolutely right. The token is clunky. The rest - a VPN to a virtual desktop - is entirely reasonable.
    - wodenokoto 3y ago
      For logging in to an online service?
      - Kalium 3y ago
        For delivering strict network access controls, a clear audit trail at multiple points, and (at a guess) the usage of very temporary credentials to actually log in, yes. Especially if the same infrastructure is used for other, less SaaS-centric data handling tasks or users are in places that like to mandate access to geo-local data (India, China, etc.). Security measures in operational workflows are only a little bit about what the end user - you - sees. Generally they're really about delivering something less obviously visible to the user but very valuable to the business, investors, and regulators.
riddlemethat 3y ago
What are your recommendations for organizations? Is there guidance for startup companies to follow?
- Kalium 3y ago
  In theory, there are documents out there a startup can read, understand, and follow to get reasonable outcomes. Theory is a wonderful place, I'm told. I'd like to visit someday. In practice, the answer is to hire a consultancy. There are fractional CISO services out there. It's very likely that a startup without significant security experience in-house does not have the expertise to make good use of any guidance they can find. Every scenario is too specific, every personality dynamic too unique, and every new technology question too novel for there to be easy pre-fab answers that a non-security-practioner can find, understand, and apply in a correct and consistent manner.
- abadpoli 3y ago
  At my company, the big thing that I think would help is if people would stop treating data engineers as if they are a cheap way to get both software engineering + data analytics in one package. They’re not. Data engineers are brilliant at data things like writing complex pipelines and wicked queries, but they just aren’t as good at writing secure code. Meanwhile, software engineers are the opposite. Data engineering teams need at least part of their team to be designated software or systems engineers that are experts in stuff like how to create a robust cloud environment. This is not only beneficial for security, but also for cost optimization, performance, operations, etc! It drives me crazy when I see a data engineer or marketing team told they have to fend for themselves when trying to figure out how to build a cloud environment.
- ziddoap 3y ago
  The NIST Cybersecurity Framework for small businesses is a great starting point for developing and nurturing a healthy cybersecurity program in a startup. https://www.nist.gov/itl/smallbusinesscyber https://www.nist.gov/itl/smallbusinesscyber https://www.nist.gov/itl/smallbusinesscyber/nist-cybersecurity-framework-0 https://www.nist.gov/itl/smallbusinesscyber/nist-cybersecuri...
markus_zhang 3y ago
You probably don't really have a proper data engineering teams -- they are most likely BI disguised as DE -- don't ask me how I know it.
reaperducer 3y ago
They inherently desire to work with the company’s Crown Jewels of data. They want unfettered access to, and want to give other people access to, an entire data warehouse of some of our company’s most sensitive customer information. In my organization, the problem is the security people, not the marketing people. They're simply not responsive. Someone on a non-IT team will send a message to someone in IT Security asking a question, checking to see if a procedure is correct, or if it's OK to so something, and the IT person simply won't get back to them in an acceptable timeframe. I'm not talking about minutes. I'm talking days. I've seen my coworkers' IT Security tickets go unanswered for more than a week. Yes, securing your big fancy data warehouse is very important. But guess what — hackers don't knock on the front door anymore. They're going through the small cracks that you allow to form when you don't realize that your job is securing the entire organization, not just your silo. This is why the rest of the company is going around security to get things done. Because if the money doesn't keep flowing, everyone loses their jobs.
- abadpoli 3y ago
  _Every single person_ in the security industry that I know is on the verge of burnout because we’re all running around trying to answer questions from other teams as fast as possible, but there just simply aren’t enough security people to go around, and I don’t think there ever will be. My security team coworkers each have a backlog of 50-100 unread slack DMs from developer teams looking for guidance at any given time, and it’s been that way for years. My team has been attempting to hire nonstop, but the pace of hiring can’t keep up with the demand. Honestly, 1 week sounds totally reasonable to me. If you’re at the point where you need an answer immediately or it’s going to delay your timelines, then you didn’t plan your timelines very well, or you waited too long to engage your security team. You have to keep in mind that while you might be laser-focused on your app and think it’s the highest priority, your security team is likely dealing with hundreds of apps, each of which all claim to be the highest priority. Triaging has to happen, and it doesn’t always go in your favor. It really sucks for everyone involved, but I don’t see a good fix for it. The sweet spot is empowering teams to make most security decisions on their own so the capacity of security people can be spent more wisely and development teams don’t have to wait for an answer except for in very rare cases, but the pessimist in me thinks we’re already too far behind on this and technology (and the opportunity for new security dangers to arise) moves too fast for us to ever catch up.
consumer451 3y ago
I have no idea if this is realistic, but maybe something like pager duty for security folks would be workable? Basically, there is always an assigned person on call to oversee the use of any "crown jewel" data? This way the internal users can continue to move fast, but maybe break way fewer things?
- abadpoli 3y ago
  IME many security people are already on call. What you’re describing is one of the roles of Security Operations Centers, unless I’m misunderstanding what you’re suggesting (which I think might be the case!) You can’t rely on that as a fix though, because it’s inherently reactive, when what you really want is proactive protections. By the time you get paged about a breach, it’s already too late.
  - consumer451 3y ago
    Thanks, I have no experience in big orgs. What I was trying to get across is a process like: 1) Marketing person wants to play with sensitive data 2) Prior to anything happening, security-aware person is notified 3) Same day, security-aware person deploys the data where the marketing person wanted it, with proper safeguards, hands-off access edit: I believe you addressed what I am getting at in your recent comment here... https://news.ycombinator.com/item?id=39995747 https://news.ycombinator.com/item?id=39995747
    - abadpoli 3y ago
      I see! I think it’s a good idea, and I personally think that would be a good and overall positive role! However in my experience in a big org, #1 happens frequently enough where it basically would be needed at all times, and would need a security person (or entire team) dedicated solely to that purpose. And sadly in my experience, companies are not willing to spend the money, or able to hire enough security people, to have that level of a dedicated security concierge for every team that needs it.
      - 3y ago
closeparen 3y ago
Centralized data warehouse. Convenient tools for querying, running arbitrary jobs against, and building reports from that warehouse. Table and column level ACLs. Strong norms and policies against copying/transferring out of the warehouse for any reason except transient UI display. Sufficient capabilities within the warehouse that it's not necessary.

Kalium 3y ago

As a security person trying to work with marketing and data engineering teams, I've often found them very resistant to learning how to secure things. They generally want to use the tools they know, in the way they know, to do the tasks they understand, on a timeline they find convenient. Anything that tries to get them to understand the risks they are taking or the sensitivity of the data, much less de-risk their workflows, is treated as an obstacle to be routed around. Often, the best I can hope for is a token effort at negotiation where their goal will be to avoid any and all changes on their part. After which I will have to monitor them carefully, because from experience the odds of them backsliding within a week are uncomfortably high. Nothing about this is conducive to producing a healthy environment. When people's idea of "easy" is they can download the company's most sensitive data to their laptop to load into Jupyter, any amount of security controls will come as an imposition.

bluedemon 3y ago
A question for you and anyone else in security: Given that many data engineers have a data science, data analytics, BI, or software engineering background, I'm curious if you've noticed any trends in their approach to data security?
- Kalium 3y ago
  Yes. Generally it can be summarized as "What data security?". Snark aside, there's usually a reflexive assumption that more data is always better and that anything that gets in the way of more data is bad. Anything that limits how data is analyzed is bad. Anything that limits or restricts their choice of tooling or where they use it is rejected. Data scientists and engineers are people who are, often, working with a company's crown jewels. They are trusted with data representing the private lives of hundreds of millions of people assembled in a data warehouse. I want them to have a care. To treat this with due gravitas. All too often, all they seem to see is a neat data set to feed into R on their Macbook.
wodenokoto 3y ago
Log in to your vpn, then log in to a website to download a token. Then use said token to log on to a virtual desktop. From here you can open a browser and log in to powerBI, which has access to data in the data warehouse. Yes, such workflows are something to be routed around.
- Kalium 3y ago
  You're absolutely right. The token is clunky. The rest - a VPN to a virtual desktop - is entirely reasonable.
  - wodenokoto 3y ago
    For logging in to an online service?
    - Kalium 3y ago
      For delivering strict network access controls, a clear audit trail at multiple points, and (at a guess) the usage of very temporary credentials to actually log in, yes. Especially if the same infrastructure is used for other, less SaaS-centric data handling tasks or users are in places that like to mandate access to geo-local data (India, China, etc.). Security measures in operational workflows are only a little bit about what the end user - you - sees. Generally they're really about delivering something less obviously visible to the user but very valuable to the business, investors, and regulators.

riddlemethat 3y ago

What are your recommendations for organizations? Is there guidance for startup companies to follow?

Kalium 3y ago
In theory, there are documents out there a startup can read, understand, and follow to get reasonable outcomes. Theory is a wonderful place, I'm told. I'd like to visit someday. In practice, the answer is to hire a consultancy. There are fractional CISO services out there. It's very likely that a startup without significant security experience in-house does not have the expertise to make good use of any guidance they can find. Every scenario is too specific, every personality dynamic too unique, and every new technology question too novel for there to be easy pre-fab answers that a non-security-practioner can find, understand, and apply in a correct and consistent manner.
abadpoli 3y ago
At my company, the big thing that I think would help is if people would stop treating data engineers as if they are a cheap way to get both software engineering + data analytics in one package. They’re not. Data engineers are brilliant at data things like writing complex pipelines and wicked queries, but they just aren’t as good at writing secure code. Meanwhile, software engineers are the opposite. Data engineering teams need at least part of their team to be designated software or systems engineers that are experts in stuff like how to create a robust cloud environment. This is not only beneficial for security, but also for cost optimization, performance, operations, etc! It drives me crazy when I see a data engineer or marketing team told they have to fend for themselves when trying to figure out how to build a cloud environment.
ziddoap 3y ago
The NIST Cybersecurity Framework for small businesses is a great starting point for developing and nurturing a healthy cybersecurity program in a startup. https://www.nist.gov/itl/smallbusinesscyber https://www.nist.gov/itl/smallbusinesscyber https://www.nist.gov/itl/smallbusinesscyber/nist-cybersecurity-framework-0 https://www.nist.gov/itl/smallbusinesscyber/nist-cybersecuri...

markus_zhang 3y ago

You probably don't really have a proper data engineering teams -- they are most likely BI disguised as DE -- don't ask me how I know it.

reaperducer 3y ago

They inherently desire to work with the company’s Crown Jewels of data. They want unfettered access to, and want to give other people access to, an entire data warehouse of some of our company’s most sensitive customer information. In my organization, the problem is the security people, not the marketing people. They're simply not responsive. Someone on a non-IT team will send a message to someone in IT Security asking a question, checking to see if a procedure is correct, or if it's OK to so something, and the IT person simply won't get back to them in an acceptable timeframe. I'm not talking about minutes. I'm talking days. I've seen my coworkers' IT Security tickets go unanswered for more than a week. Yes, securing your big fancy data warehouse is very important. But guess what — hackers don't knock on the front door anymore. They're going through the small cracks that you allow to form when you don't realize that your job is securing the entire organization, not just your silo. This is why the rest of the company is going around security to get things done. Because if the money doesn't keep flowing, everyone loses their jobs.

abadpoli 3y ago
_Every single person_ in the security industry that I know is on the verge of burnout because we’re all running around trying to answer questions from other teams as fast as possible, but there just simply aren’t enough security people to go around, and I don’t think there ever will be. My security team coworkers each have a backlog of 50-100 unread slack DMs from developer teams looking for guidance at any given time, and it’s been that way for years. My team has been attempting to hire nonstop, but the pace of hiring can’t keep up with the demand. Honestly, 1 week sounds totally reasonable to me. If you’re at the point where you need an answer immediately or it’s going to delay your timelines, then you didn’t plan your timelines very well, or you waited too long to engage your security team. You have to keep in mind that while you might be laser-focused on your app and think it’s the highest priority, your security team is likely dealing with hundreds of apps, each of which all claim to be the highest priority. Triaging has to happen, and it doesn’t always go in your favor. It really sucks for everyone involved, but I don’t see a good fix for it. The sweet spot is empowering teams to make most security decisions on their own so the capacity of security people can be spent more wisely and development teams don’t have to wait for an answer except for in very rare cases, but the pessimist in me thinks we’re already too far behind on this and technology (and the opportunity for new security dangers to arise) moves too fast for us to ever catch up.

consumer451 3y ago

I have no idea if this is realistic, but maybe something like pager duty for security folks would be workable? Basically, there is always an assigned person on call to oversee the use of any "crown jewel" data? This way the internal users can continue to move fast, but maybe break way fewer things?

abadpoli 3y ago
IME many security people are already on call. What you’re describing is one of the roles of Security Operations Centers, unless I’m misunderstanding what you’re suggesting (which I think might be the case!) You can’t rely on that as a fix though, because it’s inherently reactive, when what you really want is proactive protections. By the time you get paged about a breach, it’s already too late.
- consumer451 3y ago
  Thanks, I have no experience in big orgs. What I was trying to get across is a process like: 1) Marketing person wants to play with sensitive data 2) Prior to anything happening, security-aware person is notified 3) Same day, security-aware person deploys the data where the marketing person wanted it, with proper safeguards, hands-off access edit: I believe you addressed what I am getting at in your recent comment here... https://news.ycombinator.com/item?id=39995747 https://news.ycombinator.com/item?id=39995747
  - abadpoli 3y ago
    I see! I think it’s a good idea, and I personally think that would be a good and overall positive role! However in my experience in a big org, #1 happens frequently enough where it basically would be needed at all times, and would need a security person (or entire team) dedicated solely to that purpose. And sadly in my experience, companies are not willing to spend the money, or able to hire enough security people, to have that level of a dedicated security concierge for every team that needs it.
    - 3y ago

closeparen 3y ago

Centralized data warehouse. Convenient tools for querying, running arbitrary jobs against, and building reports from that warehouse. Table and column level ACLs. Strong norms and policies against copying/transferring out of the warehouse for any reason except transient UI display. Sufficient capabilities within the warehouse that it's not necessary.