6 ms·

Isn't this the usual practice with any nontrivial XML handling? You get some WSDL and XSDs of a coarse-grained service (as in having 300 optional input elements

by janci 5y ago

Isn't this the usual practice with any nontrivial XML handling? You get some WSDL and XSDs of a coarse-grained service (as in having 300 optional input elements with no idea what can be combined or not) so you throw real-world examples on your implementation until it works.

stouset 5y ago
"Throw stuff at it until it looks like it works" is probably not the development practice you want to employ for an authentication service.
- janci 5y ago
  Surely not! I'm just saying I'm not surprised too much.
  - jrockway 5y ago
    I think there's a mismatch between security and software engineering. Software engineers are paid to make things work, but the point of security is to make things not work (for attackers). So there is that fundamental mismatch which leads to a lot of strife, I think. (The real answer, of course, is that the software needs to do two things instead of just one. It needs to permit authorized users AND deny unauthorized users. One is not more important than the other, both must work perfectly. But when testing it, you're mostly happy when you, the authorized user, are allowed to log in, and so tend to forget about the "what if the assertion is malformed" case.) I think it's funny because engineers are always portrayed as being grumpy and that they hate everything, but if you look at how they handle errors or security, they seem like the type of people that like everything and always imagine the happy path ;)
    - 1cvmask 5y ago
      You framed the issue perfectly. A mismatch of competing incentives. That seems to be a problem everywhere beyond just security. it seems that bonuses should be tied to security as well. Then maybe software engineers would be incentivized to build things securely. A clawback of bonuses for security vulnerabilities might be an interesting idea? Or better yet bigger bonuses if there are no breaches.
      - yourapostasy 5y ago
        > A clawback of bonuses for security vulnerabilities might be an interesting idea? Or better yet bigger bonuses if there are no breaches. Agency problem. Those who decide budgets and project timelines get the upside in such a bonus scheme, while whose who implement get the downside. Clawback bonuses, and the engineers not only get blamed, they get laid off to make the quarterly numbers for the management who want to make up for lost bonuses. I suspect the solution lays in the spec complexity. Haven’t thought of this much, so my wrongthink knee jerk reaction is make a set of reference test harnesses part of the SOT of the spec, and that is the first step towards more robust implementations. Would like to hear how others have approached this problem space.
      - scrose 5y ago
        All this would accomplish is engineers feeling less like they should report issues with systems.
    - stouset 5y ago
      I've often made this comparison. Software engineering seems primarily focused on making it work correctly, security engineering is about making it fail correctly. But I also think that latter part should be just given more weight in software engineering in general. A simple example is regex: most people tend to think about writing regexes that positively match expected input, but this generally results in expressions that match things it shouldn't. Writing them is generally easier and more straightforward when you think about not matching what they shouldn't!
      - jcranmer 5y ago
        That's a good point, but I'd actually go a step further and argue that we need to think more generally about actually defining when things should fail. This is exacerbated by a tendency of many people to actually require systems to not fail on bad input but instead try to make as much sense of it as possible so that now compatibility requires you to make the same guesses as to what the user actually meant. See, for example, the tag soup parsing of HTML (which was eventually standardized in HTML5), or the insanity that is trying to parse email messages.
        mooreds 5y ago
        > This is exacerbated by a tendency of many people to actually require systems to not fail on bad input but instead try to make as much sense of it as possible so that now compatibility requires you to make the same guesses as to what the user actually meant. Isn't this Postel's law in practice? https://en.wikipedia.org/wiki/Robustness_principle https://en.wikipedia.org/wiki/Robustness_principle There's a competitive advantage in handling bad input, right? I'm not saying it should always be the primary consideration and certainly not in security situations, but I think that competitive pressures are how we arrived at this situation.
- fivre 5y ago
  and yet, "throw features at it until it meets the client's requested idealized spec, with weird carve-outs for some legacy system that we promise will be gone in a year or so, maybe" while also maybe conforming to some mythological security model drunkenly conceived by Baphomet" is where we arrive at in practice
michaelt 5y ago
It is indeed a common way of parsing XML. Unfortunately, it's a bad idea when dealing with SAML. SAML is a way for exchanging 'security assertions' - for example, if a Alice logs into the AWS Console with her employer's single-sign-on service, the SSO service gives Alice a signed document saying "The bearer of this document can log into AWS as Alice within the next 60 seconds and access accounts B and C" which she hands to AWS. The digital signature on the document is very important. Unfortunately, SAML uses the 'XML Signature' standard [1] The people who wrote the XML signature standard didn't just take an XML document and whack a PGP signature around the whole thing! They thought about use cases like 'what if you wanted to only sign part of a document' and 'what if, after signing, the document was processed by something that reordered the attributes' and 'what if you have signatures from multiple sources and want to collate them into the same document'. They decided they wanted to support all of them. As such, it's entirely legal for a document to contain a valid set of properly signed security assertions and a load of different security assertions that aren't signed. So SAML documents have to be parsed very carefully - if you naively check the document has a valid signature then read the assertions, an attacker can add extra assertions outside of the signed part of the document, granting them broader permissions than they ought to have. [1] https://en.wikipedia.org/wiki/XML_Signature https://en.wikipedia.org/wiki/XML_Signature
- bawolff 5y ago
  > The people who wrote the XML signature standard didn't just take an XML document and whack a PGP signature around the whole thing! They thought about use cases like 'what if you wanted to only sign part of a document' and 'what if, after signing, the document was processed by something that reordered the attributes' and 'what if you have signatures from multiple sources and want to collate them into the same document'. They decided they wanted to support all of them. You forgot to add the part - what if someone wants to add comments randomly without it changing the signature, even when those comments (mildly) change how the xml doc is parsed!
  - stouset 5y ago
    I did forget. I've spent years trying to.

by janci 5y ago

stouset 5y ago

"Throw stuff at it until it looks like it works" is probably not the development practice you want to employ for an authentication service.

janci 5y ago
Surely not! I'm just saying I'm not surprised too much.
- jrockway 5y ago
  I think there's a mismatch between security and software engineering. Software engineers are paid to make things work, but the point of security is to make things not work (for attackers). So there is that fundamental mismatch which leads to a lot of strife, I think. (The real answer, of course, is that the software needs to do two things instead of just one. It needs to permit authorized users AND deny unauthorized users. One is not more important than the other, both must work perfectly. But when testing it, you're mostly happy when you, the authorized user, are allowed to log in, and so tend to forget about the "what if the assertion is malformed" case.) I think it's funny because engineers are always portrayed as being grumpy and that they hate everything, but if you look at how they handle errors or security, they seem like the type of people that like everything and always imagine the happy path ;)
  - 1cvmask 5y ago
    You framed the issue perfectly. A mismatch of competing incentives. That seems to be a problem everywhere beyond just security. it seems that bonuses should be tied to security as well. Then maybe software engineers would be incentivized to build things securely. A clawback of bonuses for security vulnerabilities might be an interesting idea? Or better yet bigger bonuses if there are no breaches.
    - yourapostasy 5y ago
      > A clawback of bonuses for security vulnerabilities might be an interesting idea? Or better yet bigger bonuses if there are no breaches. Agency problem. Those who decide budgets and project timelines get the upside in such a bonus scheme, while whose who implement get the downside. Clawback bonuses, and the engineers not only get blamed, they get laid off to make the quarterly numbers for the management who want to make up for lost bonuses. I suspect the solution lays in the spec complexity. Haven’t thought of this much, so my wrongthink knee jerk reaction is make a set of reference test harnesses part of the SOT of the spec, and that is the first step towards more robust implementations. Would like to hear how others have approached this problem space.
    - scrose 5y ago
      All this would accomplish is engineers feeling less like they should report issues with systems.
  - stouset 5y ago
    I've often made this comparison. Software engineering seems primarily focused on making it work correctly, security engineering is about making it fail correctly. But I also think that latter part should be just given more weight in software engineering in general. A simple example is regex: most people tend to think about writing regexes that positively match expected input, but this generally results in expressions that match things it shouldn't. Writing them is generally easier and more straightforward when you think about not matching what they shouldn't!
    - jcranmer 5y ago
      That's a good point, but I'd actually go a step further and argue that we need to think more generally about actually defining when things should fail. This is exacerbated by a tendency of many people to actually require systems to not fail on bad input but instead try to make as much sense of it as possible so that now compatibility requires you to make the same guesses as to what the user actually meant. See, for example, the tag soup parsing of HTML (which was eventually standardized in HTML5), or the insanity that is trying to parse email messages.
      - mooreds 5y ago
        > This is exacerbated by a tendency of many people to actually require systems to not fail on bad input but instead try to make as much sense of it as possible so that now compatibility requires you to make the same guesses as to what the user actually meant. Isn't this Postel's law in practice? https://en.wikipedia.org/wiki/Robustness_principle https://en.wikipedia.org/wiki/Robustness_principle There's a competitive advantage in handling bad input, right? I'm not saying it should always be the primary consideration and certainly not in security situations, but I think that competitive pressures are how we arrived at this situation.
fivre 5y ago
and yet, "throw features at it until it meets the client's requested idealized spec, with weird carve-outs for some legacy system that we promise will be gone in a year or so, maybe" while also maybe conforming to some mythological security model drunkenly conceived by Baphomet" is where we arrive at in practice

michaelt 5y ago

It is indeed a common way of parsing XML. Unfortunately, it's a bad idea when dealing with SAML. SAML is a way for exchanging 'security assertions' - for example, if a Alice logs into the AWS Console with her employer's single-sign-on service, the SSO service gives Alice a signed document saying "The bearer of this document can log into AWS as Alice within the next 60 seconds and access accounts B and C" which she hands to AWS. The digital signature on the document is very important. Unfortunately, SAML uses the 'XML Signature' standard [1] The people who wrote the XML signature standard didn't just take an XML document and whack a PGP signature around the whole thing! They thought about use cases like 'what if you wanted to only sign part of a document' and 'what if, after signing, the document was processed by something that reordered the attributes' and 'what if you have signatures from multiple sources and want to collate them into the same document'. They decided they wanted to support all of them. As such, it's entirely legal for a document to contain a valid set of properly signed security assertions and a load of different security assertions that aren't signed. So SAML documents have to be parsed very carefully - if you naively check the document has a valid signature then read the assertions, an attacker can add extra assertions outside of the signed part of the document, granting them broader permissions than they ought to have. [1] https://en.wikipedia.org/wiki/XML_Signature https://en.wikipedia.org/wiki/XML_Signature

bawolff 5y ago
> The people who wrote the XML signature standard didn't just take an XML document and whack a PGP signature around the whole thing! They thought about use cases like 'what if you wanted to only sign part of a document' and 'what if, after signing, the document was processed by something that reordered the attributes' and 'what if you have signatures from multiple sources and want to collate them into the same document'. They decided they wanted to support all of them. You forgot to add the part - what if someone wants to add comments randomly without it changing the signature, even when those comments (mildly) change how the xml doc is parsed!
- stouset 5y ago
  I did forget. I've spent years trying to.