6 ms·

I scanned the introduction, but failed to see any concrete information on the expertise of the authors on security. Can anybody speak to the expertise of the au

by Veserv 7y ago

I scanned the introduction, but failed to see any concrete information on the expertise of the authors on security. Can anybody speak to the expertise of the authors on security? In particular, I am interested in specific projects or initiatives they directed or lead. The state of systems before and after these projects. If there were any long-term regressions after their involvement. To be even more concrete if possible: 1. What was the project and what would occur in the event of unmitigated compromise? 2. What was the threat model? 2a. Why was that the appropriate threat model given the possible outcomes? 3. How did they validate that the project met its goals in mitigating the threats in the threat model? 4. What level of resources would be necessary to compromise the systems they were trying to protect? 4a. Would the system prevent compromise by a red team with a $1 Billion, $1 Million, $1000, $1 budget? 4b. What resources did the red teams have? Personal questions for the responder: 1. Would you feel comfortable using the processes you have used in the past to develop a system where compromise would result in the loss of human life? 2. If you answered yes, what project and process and why do you believe that it sufficient? 3. If you answered no, do you have any first hand knowledge of systems that achieve that standard? 4. What is the best system that you have first hand knowledge of that has achieved at least that standard? Is there a non-theoretical gold standard?

sethvargo 7y ago
> Can anybody speak to the expertise of the authors on security? I think a cursory LinkedIn or social media search for any of the title authors or chapter authors will demonstrate their credentials. There were many people involved in this book, all of whom carry the necessary credentials and experience. > Personal questions for the responder: Guidelines help us scale, but at the end of the day, some services are unique and require additional review. I recommend reading the bits on threat modeling for more information.
- Veserv 7y ago
  Generic credentials and experience provide very little information to me on their expertise. The CSO of JP Morgan, James Cummings, was highly experienced and credentialed when JP Morgan was breached in 2014 in one of the largest data breaches in history. The CSO of Equifax, Susan Mauldin, was highly experienced when Equifax was breached. The head of security for Windows at Microsoft is probably highly credentialed and experienced, but we all make fun of the insecurity of Windows. This is why I am interested in the specific projects they worked on and how they stack up. It is much harder to game the system if there is concrete, auditable evidence backing their expertise. Yes, guidelines are not the end-all-be-all and you can never be sure, but when a civil engineer approves a bridge, they assert that they are confident that human lives can be trusted to the bridge (in certain configurations). They can do this with reasonable confidence because they have seen systems that have stood the test of time that prove out the techniques that they are applying. That is what I am interested in, do you/they have that level of confidence? What justifies that confidence? What systems prove out the techniques that were used? Did any techniques they invent stand the test of time (this provides evidence they can invent new techniques)?
AaronFriel 7y ago
> "Would the system prevent compromise by a red team with a $1 Billion... budget" Are there any such systems deployed in the world today?
- containrh4x0r 7y ago
  jp morgan chase has a security budget of $600M/yr https://www.secureworldexpo.com/industry-news/jpmorgan-chase-cybersecurity-budget https://www.secureworldexpo.com/industry-news/jpmorgan-chase...
  - Veserv 7y ago
    That is red team + blue team. Also, it makes no claims as to the effectiveness of their systems, only how much they spent. Spending money does not mean spending money effectively, if anything declaring how good you are by how much you spent is anti-correlated with quality in almost everything e.g. "I spend by far the most money out of all my friends when repairing my car." is probably more of a sign that you are getting cheated rather than high quality repairs. The other point is you actually need to successfully defend against the attackers. If you have a $100M/yr red team budget that you use to run 100 $1M red team operations and every single one compromises your systems, that does not mean you need a $100M budget to breach the systems, it means you need $1M or less. You would need to successfully defend against all operations in the year to have any confidence that your defenses are comparable to your budget.
- Veserv 7y ago
  That is the point of the question. If you believe that is not possible, then we should not use software in systems where an entity can derive more than $1 Billion in value from compromising the software otherwise it makes economic sense for them to compromise it. If the authors believe it is possible, I would like concrete information that is sufficient to counter the belief that it is not possible (I am pretty sure most people in the technical community believe it is not possible). As an example of consequences of a belief that it is not possible: The JP Morgan hack resulted in the loss of 76 million records. That means if each person's record is worth more than $14 it would be profitable for someone to hack JP Morgan if they had a system that could not prevent compromise by a red team with a $1 Billion budget. Given your question, I will assume you do not believe that such a system does not exist, in fact, you probably believe there is no system that is even in the general vicinity of that number (apologies if I am misinterpreting your statement). If we assume a $1 Million budget is all it takes, then each record would only need to be worth 1.4 cents for it to be profitable to hack JP Morgan. How do you think people would feel about that? Do you think it would be problematic for JP Morgan if they announced they protect every account with $14 of security, let alone 1.4 cents? Now the JP Morgan hack is a little old, it happened in 2014, so let's use something newer. In 2019 Wells Fargo lost over 24 million records. At $1 Billion that is $40 per record. At $1 Million 4 cents. Do you think it would be problematic for Wells Fargo if they announced they protect every account with $40 of security, let alone 4 cents?
jwkane 7y ago
1. Is it reasonable to expect a very specific list of questions to be addressed in the introduction to a book?
- Veserv 7y ago
  No. These are questions for people in this thread who may have knowledge on these matters. They are also just detail for the more general question asked earlier about their expertise in case whoever is reading could speak to those more specific questions.
tasuki 7y ago
You seem much more interested in the authors than in the book.
- Veserv 7y ago
  Indeed I am. Would you trust the contents of a book on a technical topic if the authors are not, in fact, subject matter experts? Would you read a book on cancer treatment by a doctor of theology with no medical training? To use a less egregious example, a neurologist with no training in oncology or experience with brain cancer? Knowing the expertise of the authors is very important, especially if you are not a subject matter expert in your own right who can directly evaluate the claims made and methodologies used. This is why such books usually have a little blurb which explains why the author is a subject matter expert. I go one step further in asking for third or first-party confirmation of expertise in this interactive forum because self-made blurbs are easy to manufacture, and credentials and experience can be gamed. It is much harder to distort the opinions colleagues or people who use their systems every day after they have left. I ask the more detailed questions because I find that answers to general questions are usually pretty wishy-washy and thus not particularly useful. The detailed questions ask for more detailed metrics and observations that make it easier to sift out useful information. For instance, I ask what projects they lead or directed since I want to filter out cases where they may have been on important projects, but in an unimportant role or possibly even being carried by the rest of their team.
  - apetresc 7y ago
    But why this unusual level of scrutiny? The book is published and endorsed by both Google and O'Reilly, two of the most respected brands in this domain. Why are they not a satisfactory "third or first-party confirmation of expertise" but random commenters on Hacker News would be?
    - Veserv 7y ago
      Because I do not respect the brands or general reputation, so I would like to know if there is substance underlying the reputation. If the reputation is built on truth instead of marketing, I would hope that there are examples of substance where they achieve what their reputation says they can do. I believe this an appropriate general strategy if one does not believe in the reputation, so this is not really an unusual level of scrutiny, it is just that I do not believe in the brand where as you do (not that there is anything wrong with that). As for random commenters, since I can not use the reputational apparatus for information, I would need to get direct information. There are likely people here who work with the stated individuals, so there is a non-zero chance I could get that information. Hopefully random commenters would not lie for no reason, but to deal with that I avoid incorporating information that can not be cross-referenced.
MaxBarraclough 7y ago
> Would you feel comfortable using the processes you have used in the past to develop a system where compromise would result in the loss of human life? I don't think that's a fair question. Life-or-death software systems (such as avionics) are built using very different methodologies than ordinary software, and their development costs (and times) are orders of magnitude higher. If ordinary software were held to the same standard, almost no software development projects would pay for themselves.

by Veserv 7y ago

sethvargo 7y ago

> Can anybody speak to the expertise of the authors on security? I think a cursory LinkedIn or social media search for any of the title authors or chapter authors will demonstrate their credentials. There were many people involved in this book, all of whom carry the necessary credentials and experience. > Personal questions for the responder: Guidelines help us scale, but at the end of the day, some services are unique and require additional review. I recommend reading the bits on threat modeling for more information.

Veserv 7y ago
Generic credentials and experience provide very little information to me on their expertise. The CSO of JP Morgan, James Cummings, was highly experienced and credentialed when JP Morgan was breached in 2014 in one of the largest data breaches in history. The CSO of Equifax, Susan Mauldin, was highly experienced when Equifax was breached. The head of security for Windows at Microsoft is probably highly credentialed and experienced, but we all make fun of the insecurity of Windows. This is why I am interested in the specific projects they worked on and how they stack up. It is much harder to game the system if there is concrete, auditable evidence backing their expertise. Yes, guidelines are not the end-all-be-all and you can never be sure, but when a civil engineer approves a bridge, they assert that they are confident that human lives can be trusted to the bridge (in certain configurations). They can do this with reasonable confidence because they have seen systems that have stood the test of time that prove out the techniques that they are applying. That is what I am interested in, do you/they have that level of confidence? What justifies that confidence? What systems prove out the techniques that were used? Did any techniques they invent stand the test of time (this provides evidence they can invent new techniques)?

AaronFriel 7y ago

> "Would the system prevent compromise by a red team with a $1 Billion... budget" Are there any such systems deployed in the world today?

containrh4x0r 7y ago
jp morgan chase has a security budget of $600M/yr https://www.secureworldexpo.com/industry-news/jpmorgan-chase-cybersecurity-budget https://www.secureworldexpo.com/industry-news/jpmorgan-chase...
- Veserv 7y ago
  That is red team + blue team. Also, it makes no claims as to the effectiveness of their systems, only how much they spent. Spending money does not mean spending money effectively, if anything declaring how good you are by how much you spent is anti-correlated with quality in almost everything e.g. "I spend by far the most money out of all my friends when repairing my car." is probably more of a sign that you are getting cheated rather than high quality repairs. The other point is you actually need to successfully defend against the attackers. If you have a $100M/yr red team budget that you use to run 100 $1M red team operations and every single one compromises your systems, that does not mean you need a $100M budget to breach the systems, it means you need $1M or less. You would need to successfully defend against all operations in the year to have any confidence that your defenses are comparable to your budget.
Veserv 7y ago
That is the point of the question. If you believe that is not possible, then we should not use software in systems where an entity can derive more than $1 Billion in value from compromising the software otherwise it makes economic sense for them to compromise it. If the authors believe it is possible, I would like concrete information that is sufficient to counter the belief that it is not possible (I am pretty sure most people in the technical community believe it is not possible). As an example of consequences of a belief that it is not possible: The JP Morgan hack resulted in the loss of 76 million records. That means if each person's record is worth more than $14 it would be profitable for someone to hack JP Morgan if they had a system that could not prevent compromise by a red team with a $1 Billion budget. Given your question, I will assume you do not believe that such a system does not exist, in fact, you probably believe there is no system that is even in the general vicinity of that number (apologies if I am misinterpreting your statement). If we assume a $1 Million budget is all it takes, then each record would only need to be worth 1.4 cents for it to be profitable to hack JP Morgan. How do you think people would feel about that? Do you think it would be problematic for JP Morgan if they announced they protect every account with $14 of security, let alone 1.4 cents? Now the JP Morgan hack is a little old, it happened in 2014, so let's use something newer. In 2019 Wells Fargo lost over 24 million records. At $1 Billion that is $40 per record. At $1 Million 4 cents. Do you think it would be problematic for Wells Fargo if they announced they protect every account with $40 of security, let alone 4 cents?

jwkane 7y ago

1. Is it reasonable to expect a very specific list of questions to be addressed in the introduction to a book?

Veserv 7y ago
No. These are questions for people in this thread who may have knowledge on these matters. They are also just detail for the more general question asked earlier about their expertise in case whoever is reading could speak to those more specific questions.

tasuki 7y ago

You seem much more interested in the authors than in the book.

Veserv 7y ago
Indeed I am. Would you trust the contents of a book on a technical topic if the authors are not, in fact, subject matter experts? Would you read a book on cancer treatment by a doctor of theology with no medical training? To use a less egregious example, a neurologist with no training in oncology or experience with brain cancer? Knowing the expertise of the authors is very important, especially if you are not a subject matter expert in your own right who can directly evaluate the claims made and methodologies used. This is why such books usually have a little blurb which explains why the author is a subject matter expert. I go one step further in asking for third or first-party confirmation of expertise in this interactive forum because self-made blurbs are easy to manufacture, and credentials and experience can be gamed. It is much harder to distort the opinions colleagues or people who use their systems every day after they have left. I ask the more detailed questions because I find that answers to general questions are usually pretty wishy-washy and thus not particularly useful. The detailed questions ask for more detailed metrics and observations that make it easier to sift out useful information. For instance, I ask what projects they lead or directed since I want to filter out cases where they may have been on important projects, but in an unimportant role or possibly even being carried by the rest of their team.
- apetresc 7y ago
  But why this unusual level of scrutiny? The book is published and endorsed by both Google and O'Reilly, two of the most respected brands in this domain. Why are they not a satisfactory "third or first-party confirmation of expertise" but random commenters on Hacker News would be?
  - Veserv 7y ago
    Because I do not respect the brands or general reputation, so I would like to know if there is substance underlying the reputation. If the reputation is built on truth instead of marketing, I would hope that there are examples of substance where they achieve what their reputation says they can do. I believe this an appropriate general strategy if one does not believe in the reputation, so this is not really an unusual level of scrutiny, it is just that I do not believe in the brand where as you do (not that there is anything wrong with that). As for random commenters, since I can not use the reputational apparatus for information, I would need to get direct information. There are likely people here who work with the stated individuals, so there is a non-zero chance I could get that information. Hopefully random commenters would not lie for no reason, but to deal with that I avoid incorporating information that can not be cross-referenced.

MaxBarraclough 7y ago

> Would you feel comfortable using the processes you have used in the past to develop a system where compromise would result in the loss of human life? I don't think that's a fair question. Life-or-death software systems (such as avionics) are built using very different methodologies than ordinary software, and their development costs (and times) are orders of magnitude higher. If ordinary software were held to the same standard, almost no software development projects would pay for themselves.