12 ms·
Our Dumb Security Questionnaire
- blakesterz 6y ago"But unfortunately, they’re also kinda the best we can do. Huge companies might be able to afford real human-powered security audits and convince vendors to allow them, but small startups like ours don’t have a better option than relying on DSQs." Not what I expected based on this title. I liked this. I'm not sure how many people know about the Higher Education Community Vendor Assessment Toolkit, but if you're into Dumb Security Questionnaires, this is worth checking out. "The HECVAT is a questionnaire framework specifically designed for higher education to measure vendor risk." I know it says Higher Ed, but it works just about anywhere. https://library.educause.edu/resources/2020/4/higher-education-community-vendor-assessment-toolkit https://library.educause.edu/resources/2020/4/higher-educati...
- karmakaze 6y agoThis is great. I like the open ended phrasing, there's no on a scale of 1 to 10 thoughtless responses possible.
- cinbun8 6y agoI really loved this list and think its a great place to start . Add to this a network diagram; arch diagram; Tenancy; WAF + FIM + SIM; Bring your own keys; and you can start bringing in the enterprises into your deals. Not that you need all of those, but the add-ons usually build on the struts mentioned on that list.
- alg0 6y agoBasic information about these things is alright to pass on but this is kinda information that should be a bit private. Even if here ( and anywhere on Internet ) is a lot of people that think obsecurity is bad it is important tool in any security stack. "Security by obsecurity is bad" means that there is no other security but a obsecurity. Even NIST gives a recomendation for obsecurity if your other security stack is good
- orestis 6y agoThis is indeed much shorter than what I’ve seen around. As a developer/team lead that might need to answer these questions to a satisfactory degree, what are resources that would actually help implementing this kind of security infrastructure?
- jacobian 6y ago(Author here.) Well, you could always use this questionnaire as a starting point itself: ask yourself these questions, and if you're not happy with the answers, do something about it. Another reasonable security practices starting point would be another article by Latacora: https://latacora.micro.blog/2020/03/12/the-soc-starting.html https://latacora.micro.blog/2020/03/12/the-soc-starting.html It's semi-oriented towards SOC2, but every item on that list is practical, doable even for small teams, and has real solid security impact.
- orestis 6y agoOoh, I’ve done that, and I’m doing it with many such questionnaires I receive :) sometimes it makes sense and we do something about it, but many times you just don’t know what you don’t know, or you don’t know where to start, and it’s not a topic that comes up often on the various public fora. I was looking for books, talks, guides - anything. I just read the latacora soc2 guide and it’s at least a starting point.
- gwbas1c 6y agoThis also looks like a good way to evaluate a potential employer.
- tr0ut 6y agoAre there any legal issues if one were to be untruthful/lie on security questionnaires?
- munchbunny 6y agoWhat I'm familiar with is that you would write into any business contracts signed with that vendor that all of the [insert scoping modifier here] representations the vendor has made are truthful or else the customer may cancel the contract and seek [consequences of contract violation]. Just make sure the scope includes that security questionnaire.
- Wowfunhappy 6y agoIf nothing else, I imagine if the vendor were to get hacked, the client would have obvious grounds for a lawsuit. Obviously you'd rather it not get to that point, but, still.
- ndiscussion 6y agoIANAL but wouldn't that be straightforward fraud?
- tr0ut 6y agoI asked because I've heard more than once that a company either stretched the truth or outright lied on these questionnaires. So stretching the truth could be: Do you adhere to NIST? The truth could be: "well not exactly but that's on our roadmap,we do somethings that are close enough." That would get a 'YES' check. Or something like end to end encryption. The answer could be a 'YES' because a company uses front-end TLS and pretends to not completely understand the ask. In this case it is mostly the business either forcing security to bs or another group (Sales?) filling out the response untruthfully because they are loosing revenue if they're honest.
- theevilsharpie 6y agoIf you're hacked and leak your customer's data, and it turns out that you materially lied on your customer's security questionnaire/due diligence, you could be sued by your customer for damages, and your insurance company could refuse to defend you.
- jamiesonbecker 6y ago> If you don’t use a cloud provider: why not? This makes it sound like it's a negative, but a solid argument can be made controlling your application all the way down to the metal is a more, not less, secure way to go. We can all make opinionated assumptions about whether a cloud provider is "more" or "less" secure than doing it (whatever "it" is, exactly) on your own, but even the fact that it's so debatable proves that it's not really a defining point that should be on this list of questions (although it might be to an individual team, especially if you're trying to bring things in-house in the first place, but of course this set of questions is really relevant to SaaS.)
- count 6y agoI would suggest that for the VAST majority of organizations, a cloud provider[1] will be an unequivocally more secure infrastructure than running it themselves 'down to the metal'. The big clouds have security research security ops teams larger than many companies. I don't think anyone who has seriously spent time looking at this would debate it at all. Nb. this has no bearing on if it's the right business decision, mind you. That absolutely could end with you being better on your own metal! But you won't be more secure. Source: spent a decade doing security evaluations of cloud providers for the DoD/IC, who also had this misconception going into things. [1] to be fair, I'm only talking about the large cloud providers (AWS, Azure, GCP, etc.). There are tens of thousands of SaaS/smaller cloud providers that are absolutely debatable around security posture.
- jamiesonbecker 6y ago> But you won't be more secure. I respectfully disagree. A cloud has many more vectors for attack, and even the largest clouds have blind spots. For example, check out AWS's (lack of) response to a vuln report that I filed with their security team: https://github.com/aws/aws-codedeploy-agent/issues/30 https://github.com/aws/aws-codedeploy-agent/issues/30 Did you know that AWS does NOT pay for security vulns and does not operate a bug bounty program? It seems a common myth that AWS is more secure than any particular company. Your security inside or outside of a cloud has to do more with your internal security posture than what AWS brings to the table (and leaves behind). The public cloud is not magic security pixie dust. It's a virtualization platform with extra services attached. You are still 100% responsible for securing your servers (meaning everything down to the network routing layer, and some beyond that, too). Obligatory reference to the AWS Shared Responsibility Model[0] This is not a bad thing, but it's still a thing. Anyone who migrates to the cloud and magically expects their security posture to improve one iota is simply wrong unless they take active steps to integrate as tightly as they can into that cloud's security tools. As someone who has done a ton of AWS security audits, I can tell you that people take multiple approaches on that: either they don't integrate into it fully, or they submit to vendor lock-in. This is perhaps too deep of an argument, since clearly intelligent and skilled people can reasonably disagree. FWIW, lapsed AWS SA, recovering Fortune 50 cloud security architect, and please don't take my argument about the cloud to indicate that I'm anti-cloud: my security-focused SSH key management company, https://userify.com https://userify.com, is strongly cloud focused: AWS Advanced Tier, GCP Partner, SOC-2, PCI, HIPAA. I'm very much pro-cloud, while recognizing that it enlarges your threat model but brings other security advantages. Where you're hosted doesn't matter as much as whether you're paranoid and clueful, and perhaps have former blackhats on your team ;) So my point is simply that the cloud question isn't that relevant, and the thrust of the other questions should be used to determine security competency instead of where they're running their operations. 0. https://aws.amazon.com/compliance/shared-responsibility-model/ https://aws.amazon.com/compliance/shared-responsibility-mode...
- count 6y agoThese are pretty good, but one problem for 'general' adoption of something like this: Most customer orgs aren't qualified to judge the vendors answers because they aren't good at security either. Are you confident you know what a good answer to 'how do you push code to production' looks like?
- jacobian 6y agoI tried to get at that in the "who this is for" section: > [This assumes that] you have a security person (or team) who can evaluate the answers and is part of the decision-making process. If nobody’s going to read the answers, don’t waste your vendor’s time. So yeah, totally agree. If you can't adequately evaluate the answers, it's not worth anyone's time to ask 'em.
- count 6y agoI missed that line, sorry!
- tolstoshev 6y agoOne reason to us SaaS - if there's a hack you can blame them rather than taking the heat yourself. It's outsourcing of culpability.
- scooble 6y agoTo a point. I’ve been asked about the criteria used to evaluate third party suppliers, for example. Or the physical security measures at outsourced data centres.
- jamiesonbecker 6y agoIn an ideal world, these are mostly reasonable questions, even if a bit specific to the OP's requirements, but unless you are purchasing SaaS software on a large, enterprise-style license (i.e., 6 figures or more), it probably won't be cost-effective for most vendors to gather and collate this documentation, as well as answering these questions, just like it's not worth it to ink a custom contract on a smaller deal, where legal fees alone might eat up the entire profit margin or even push it negative. Engineering time is expensive and some of these questions are technical or "special" in nature. There's a reason why standardized third-party audits like SOC-2's and ISO-27001's exist: to reduce the time required to document your veracity and security as a vendor for a potential customer. Since even large customers rely on the statements (independently audited, unlike in these questions) of a security attestation to make purchases, why should a customer request that extra time should be taken away from other responsibilities, like making better products or providing customer support? I freely admit that I'm a bit biased against ad hoc security questions, even though I used to do it myself when working on large security teams. ;) My security-focused SSH key management company, https://userify.com https://userify.com, went through the time and expense to achieve AICPA SOC-2 certification from an independent third-party auditor to reduce the time and costs involved in responding to smaller RFP's, and to provide fully documented, standardized, and legally binding proof of our security bonafides. We still try our best to intelligently respond to any and all questions, especially about security, from any customers at all, even free-tier customers and hobbyists, but it's harder to do that when presented with a big list of questions that are mostly answered in our SOC-2 audit already.
- bostik 6y agoAt the risk drawing fire from everyone, I don't really think ISO27001 is a security certification. We get audited annually against 27k Annex A (basically ~95% of full one) + UK Gambling Commission extras. The audit focuses far more on technical aspects of business continuity than actual security. There's certainly plenty of overlap, but other than the parts about access controls and "who watches the watchmen" aspect, ISO27k is almost entirely about your ability to recover from even the most devastating disaster. The pragmatic security parts have a bolted-on feeling to ensure the recovery path remains largely uncorrupted.
- coldcode 6y agoI worked for a HIPAA covered healthcare company that routinely passed audits. They also stored all the production database and server passwords in a plain text file accessible to half the company and no audit trail of any kind on logins. Clearly no auditor ever asked or looked.
- jpcosta 6y agowhere would one hire an auditor like this? asking for a friend
- et-al 6y agoI know you're being facetious, but any SOC2 audit would overlook this because they're just going down a checklist making sure specific controls are in place and not actually probing for the various possibilities to bypass these controls.
- jamiesonbecker 6y ago> any SOC2 audit would overlook this because they're just going down a checklist From the GP, "They also stored (1) all the production database and server passwords in a plain text file accessible to half the company and (2) no audit trail of any kind on logins." Any competent SOC-2 auditor would not overlook this. Large swaths of criteria are specifically geared to uncover both of these (unencrypted credentials and audit trails). SOC-2 audit firms have a strongly vested interest in hiring competent auditors, because if a SOC-2 auditor did not ask questions covering these areas, then failure to complete the audit would be actionable malpractice and the auditor would be liable, unless the company lied (committed demonstrable fraud) in its written responses.
- DiabloD3 6y agoGiven the current political climate, such a SOC-2 auditor should be outed by name.
- _wldu 6y agoSome industries have already matured and standardized this concept. The HECVAT is one example. https://library.educause.edu/resources/2020/4/higher-education-community-vendor-assessment-toolkit#tools https://library.educause.edu/resources/2020/4/higher-educati... The benefit of this is that if you (as a vendor) wish to sell in this industry, you probably only need to complete this once, rather than one for each potential customer.
- gneray 6y agonice, thanks for sharing
- deleted 6y ago[deleted]
- motohagiography 6y agoCan do it in 4 questions: 1. What do you need to protect? 2. What is the complete list of tech stack technologies it depends on? 3. What are you doing to protect that important thing today? 4. Who do you need to protect it from? If you can't answer these, the answers to the rest of the questions really don't matter.
- lawnchair_larry 6y agoWhat would you consider a disqualifier for question 2?
- cmeacham98 6y ago- "I don't know, nobody understands that legacy system from 10 years ago" - Very outdated and/or vulnerable software - Backups of customer data are stored unencrypted on dropbox - C/C++ (ok this one is a joke)
- motohagiography 6y agoThe way it works is if you don't have the answer, you collect what you can know and move with that. Also C/C++ means you need a linter, a code review process, VAs, and potential exposure to FOSS libraries. This method is mainly for building things, but if they are legacy, the information should be available as well. The top level answer of "nobody understands this," is pretty much the most important thing you need to know from a security perspective. It is funny, but it's funny because it's true and important.
- Nerada 6y agoThese are often knocked back with "We charge $4,000 to answer Security Questionnaires, or you can download our pre-made pack [that answers none of these questions] here."
- glaucon 6y agoWhat is a "pre-made pack" ? An overview of your security stance which is fundamentally fluff ? Thanks.
- lawnchair_larry 6y agoWhich vendors charge for their DSQ?
- jacobian 6y agoAuthor here: weird, I've never experienced that (10+ years doing this silly exercise, on both sides). Various vendors have offered their own compliance frameworks - PCI reports, SOC2, whatever -- and I'm happy to read those instead; they tend to have (most of) what I'm looking for. I've never been charged for the pleasure, though. Guess I have something to look forward to!
- mdb333 6y agopretty straightforward, but does it scale? and can the answers be captured objectively in order to inform consistent decision making? i.e. this is fine for a one shot review, but would be tough to operationalize... Plus, keep in mind diagrams can be critical. If you're going to be sharing sensitive data with this vendor you're going to need to know and have documented how that data will flow, where it will persist, etc. Can be a lot easier to capture in a diagram than narrative format. There is variety of other FOSS type stuff out there that is useful for anyone that needs more: Vendor Security Alliance -- https://www.vendorsecurityalliance.org/downloadQuestionaire https://www.vendorsecurityalliance.org/downloadQuestionaire (disclaimer: I'm an advisor @VSA) CSA CAIQ -- https://cloudsecurityalliance.org/artifacts/consensus-assessments-initiative-questionnaire-v3-1/ https://cloudsecurityalliance.org/artifacts/consensus-assess... SIG --NM, looks like this is closed/member only now, but if you can track it down (SIG Lite/Full, Standard Information Gathering) its ridiculously comprehensive. Google VSAQ -- https://github.com/google/vsaq https://github.com/google/vsaq As someone who deals with both sides (asking the questions to vendors, and answering them for prospect customers) I can say they mostly all suck pretty hard... and that's probably why there's a whole ecosystem of vendors in this space nowadays.
- lawnchair_larry 6y agoAs someone who works in security and deals with these questionnaires a lot, I don’t have any idea how they caught on. It is my opinion that they mean nothing and are worth nothing. Asking how a company’s development practices protect against the OWASP Top 10 doesn’t make a lot of sense. The top 10 isn’t a static list, nor is it backed by any real data, and many of the entries are fairly generic concepts that aren’t something that can be addressed holistically with any particular development practice. Everybody lies on these. They’ll just say a best practice, or they’ll quickly run a scanner in response to the question, or they give you the policy that no one actually follows, or they’ll give a partial truth, like saying that they use 2FA, when they only use 2FA in some places, etc. Every company you hear about getting hacked had great answers on their security questionnaire. I guarantee that Equifax had a policy of patching vulnerabilities for example. These are just paperwork that wastes everyone’s time. Certifications like SOC 2 are even worse, but they sure do make some people a lot of money.
- KirillPanov 6y ago> I don’t have any idea how they caught on. Probably the lawyers asked for it. As in, if $VENDOR gets hacked and we did no due diligence, we are definitely liable for $VENDOR's incompetence. If we can prove we attempted some form of due diligence, with a paper trail, we might have a fighting chance.
- McCaffery 6y agoTotally agree with you. However, in my head it was, "hey lawyer, what should we do to protect ourselves against breaches by our vendors?" Lawyer thinks, well, we do due diligence for M&A, financings etc. etc. so why not for onboarding vendors. Course, now this process is codified into law. I have an idea for a better way... :)
- lawnchair_larry 6y agoThe thing is, lawyers aren’t asking for these. It’s mostly security bureaucrats who approve software purchases or SaaS agreements. Someone will say, how are we making sure that we can trust our vendors? Someone else will say, lets hire someone to put in charge of making sure we buy the right things. Someone who claims to be a security expert. A “security expert” says, we will ask them for their audit reports. As an expert, I will read them and tell you if they are secure. It’s just paper pushing.
- ryanSrich 6y agoHIPAA isn’t a certification just FYI. If someone is selling you a HIPAA certification they are scamming you. HIPAA “compliant” simply means you’ve hired a 3rd party to audit your business. Essentially they ensure you do what you say you do in your policies and procedures. There are certifications that can effectively illuminate the need for these types of questionnaires. HITRUST as an example in Healthcare and SOC 2 for broader SaaS providers.
- dragonwriter 6y ago> HIPAA isn’t a certification just FYI. There are HIPAA certifications, but they aren't legally required or beneficial. They may have some marketing value. > HIPAA “compliant” simply means you’ve hired a 3rd party to audit your business No, it means you are following HIPAA requirements; 3rd party audit is not a HIPAA requirement any more than certification is, though it may be useful in identifying compliance issues.
- ryanSrich 6y agoSemantics. I’m saying a HIPAA certification from HHS or any official governing body doesn’t exist. A 3rd party auditor might give you a “certification” but it’s meaningless to call yourself “HIPAA certified”. You wouldn’t say that. You’d say “we’ve conducted a 3rd party independent audit”. You can go ahead and claim HIPAA compliance all you want, but most health systems are going to ask you if you’ve conducted an audit or if you have any certifications - SOC 2, HITRUST, etc. So in reality you’re not HIPAA compliant until you can prove it. That’s my point.
- dragonwriter 6y ago“HIPAA compliant” means you aren't breaking the law, which is quite important independent of any audit or certification because there are penalties for breaking the law, and they apply to all covered entities. “HIPAA certified” and the identity of the certifying authority is also potentially important; it may have some marketing value, but more importantly it is (as you note, but bizarrely use it to propose that “audit” is more important than “certification”), various people you might want to do business with may require certification, by an acceptable certification authority. (Possibly different ones for different aspects of HIPAA.) OTOH, there are covered entities that don't have to worry about this at all, because of who they are and who they plan to do business with. A 3rd party audit for HIPAA compliance probably isn't important to anyone external to you directly (and is not an element of compliance), but may be useful for identifying compliance issues, or may be a component of certification.
- TruthWillHurt 6y agoIt really is dumb. As if a penetration test done in the last n months is proof anything is actually secured. and pointing at the OWASP top 10 and saying "do this" just screams IMPOSTER.
- permille42 6y agoQ: If you use a cloud provider (GCP/AWS/Azure/etc) A: I don't use a cloud provider Q: Describe how credentials are provisioned, managed, and stored. A: Credentials to what exactly? Since I don't use a cloud provider all the data including credentials are safe and secure on the server hosting everything. Q: If an attacker gained access to an individual developer’s cloud credentials A: The only developer is me. Anything besides what I trigger myself I would notice immediately as out of the ordinary. Q: What actions could that attacker perform? A: Not much. Externally facing applications are containerized and have access only to the level of access to data that they should. They would not gain any control of the system Q: How would you detect and respond to the breach? A: I'd notice activity in the logs which are monitored for unusual activity. I also check them manually myself besides the automatic monitoring rules. Q: If you don’t use a cloud provider: why not? A: They are overpriced and I don't trust them. Q: Describe how staff authenticate to company services (e.g. servers, email, SaaS products), particularly highlighting your use of password managers, 2FA, and SSO. A: Passwords? What b.s. is that. Certs all the way. Q: What development practices do you use to protect against the OWASP Top 10? 1. Injection. A: Only parameterized queries are used. Non-issue. 2. Broken auth. A: Only use reliable session/auth stuffs and/or look carefully at the details of how it works. Don't use crap. 3. Sensitive data exposure. A: APIs only expose the data needed and all access is logged and monitored. 4. XML External Entities. A: I don't use crappy XML parsers with such vulns. 5. Broken access control. A: All applications I run were written by myself and access levels only have access to just the needed data. That access level flows to all places data is accessed. Nothing is given access without an associated current valid session and auth level. 6. Security misconfiguration. A: System is hardened. Containers are used. Everything is kept up to date to the bleeding edge at all times. No legacy crap is used. 7. XSS. A: All access is in house. No external entities are loaded period. I trust no one. All entered data is carefully output in a way to prevent user entered data from having influence on code. Eg: No user data is trusted and everything is validated as being, essentially, plaintext. 8. Insecure deserialization: A: No outside entities are deserialized since I don't accept any. Anything that could have been altered by a user, the parsers are all secure state machines which cannot possibly have any influence on the system. 9. Using Components with Known Vulnerabilities. A: I trust nothing, including libraries. I review the code of many many many things. Obviously cannot review everything but I at least choose libraries that are kept up to data and regularly worked on / used by others. I am sure there are 0days present that aren't public yet, but even if they were taken advantage of they wouldn't get access to much. 10. Insufficient Logging & Monitoring. A: All access and operations that affect data are logged. Read access is also logged and monitored to some extent. Describe the steps a developer or operations person takes to push new code to production. Have you had any security breaches in the last two years? If yes: please explain the breach, and provide copies of any postmortem/root cause analysis/after-action reports.
- artjomb 6y agoA part of my work tasks consists of reviewing answers to security questionnaires. These are reasonable questions and I see quite a lot of value if they are filled out extensively and in a good faith approach. Most of the answers of usual security questionnaires can be deduced from the responses to this DSQ. I really have a problem with Q6: > Have you had any security breaches in the last two years? > If yes: please explain the breach, and provide copies of any postmortem/root cause analysis/after-action reports. Almost nobody will answer this truthfully. I see a couple of options: 1. There was a breach and it was public, then why are you asking. Do your research! 2. There was a breach and it was not made public. The company will likely not admit it to you. 3. There was a breach but it was a) not relevant to your case/b) internal/c) the data lost was not customer data/d) we forgot that there was/etc. While lying in case of 2. might make the vendor liable (IANAL) they might be able to argue that 3 was actually the case.
- tamiral 6y agowe have a document we ask our vendors to fill out then our security, risk and project team meets with the vendor and their respective teams to go through in detail and then we create a risk document for approval or workarounds where the vendor doesn't meet a requirement. Usually it's just like oh... you dont do this basic security... ok bye.