6 ms·

On top of this, I have inside knowledge that some extremely common libraries that implement SAML were built not by reading and understanding the spec, but simpl

On top of this, I have inside knowledge that some extremely common libraries that implement SAML were built not by reading and understanding the spec, but simply by looking at sample XML documents in the wild and writing code that handled it. Multiple exceedingly obvious vulnerabilities have been the result. One fun one was: looking at an XML signature in the document, verifying it, then ignoring the assertion it was claiming to sign and just trusting the assertion at the document root. I tried to write a standards-based implementation and gave up. The standard is enormous, and consists of three parts: 1. The definitions of what each XML tag means in a vacuum 2. Patterns on how to assemble those XML tags into a document that means something useful 3. Protocols that exchange these documents back and forth to accomplish some authentication objective Half the problem comes from the fact that it's meant to do anything and everything, and so you can theoretically just mix and match all the above parts to get what you want. But that also means that it's exceedingly simple to mix and match stuff in ways that are subtly (or not so subtly) insecure. The other half comes from the fact that the standard is so damned complicaed in order to handle everything under the sun that it's damn near impossible to wrap your head around it all. So people just glance at the spec occasionally and just write something that handles documents they see in the wild and hope for the best, with predictable outcomes. The whole thing is a tire fire. Note, I last worked with it about a decade ago so I may have gotten some of the characterizations wrong.

lol768 5y ago
> On top of this, I have inside knowledge that some extremely common libraries that implement SAML were built not by reading and understanding the spec, but simply by looking at sample XML documents in the wild and writing code that handled it. Yup, and its complexity leads to something of a shortage of libraries in the first place so there's not much choice. There's still no good modern maintained OSS .NET 5 library for doing SAML. There are some commercial offerings (ComponentSpace have a neat product and they weren't affected by VU#475445 either) but my worry is that folks will end up using whatever free libraries they can find without thinking too much about the consequences.
- 8mcoLwmQoWZ3s 5y ago
  By "doing SAML" do you mean from the server perspective or the client perspective? This simple client library works with .NET 5. https://github.com/jitbit/AspNetSaml/ https://github.com/jitbit/AspNetSaml/
  - mwcampbell 5y ago
    If I'm not mistaken, the correct terms are identity provider and service provider, respectively. After all, they're both servers, in the sense that they both receive requests from the other.
    - checker 5y ago
      The coolest part imo is that the the servers aren't communicating directly to one another. The client brokers the communication so that both IDP and SP can be on completely different networks (bridged by the client). You probably already knew this but sharing because I find it interesting (and for the edification of those that are unfamiliar).
  - stouset 5y ago
    This client library is nowhere near enough by itself. I could tell this within five seconds, as it's a single source file only a few hundred lines long. It checks that you get a signed assertion back, which is valid, and lets you pull out attributes. That's great, but there are a gajillion tags that can be set within an assertion that have genuine, important security implications and zero of them are handled by this library. If your IdP sends those tags along with the assertion, this library will happily ignore them unless you implement support yourself. And I would bet heavily that approximating zero of this library's users write that support. One trivial example, there's a `<saml:Conditions>` tag. This tag can have all sorts of attributes like `NotBefore` or `NotOnOrAfter` that specify a time period for which the assertion is valid. This library does not do anything to implement support for `<saml:Conditions>` tags at all, so every consumer of this library will happily process assertions as valid even if they're outside of the duration for which the IdP is claiming it's valid. If I get my hands on an assertion at some point in time, I can hold onto it and replay it forever to unsuspecting users of this library. There are other attributes that can set conditions, and there are other tags with implications on whether or not an assertion should be considered valid. None of them are handled. Further, only minimal document validation is performed. The signature is checked, but the Issuer is not. Multiple Issuers can share a signing key, though I suspect actual cases of this are rare. At any rate, there's a billion ways users of this library can blithely chug along with an assertion that isn't valid and shouldn't be trusted, but nobody notices or cares because the happy path works. With authentication, yes it's important that legitimate users are able to sign in. But it's as important that illegitimate authentication attempts are disallowed. And this half is completely forgotten about in literally 100% of the common SAML service provider libraries I've seen in the wild (which is admittedly not 100% of the ones available, just the ones I've looked at). Edit: As @tptacek points out, every user of this library is trivially susceptible to AudienceRestriction authz vulnerabilities. An IdP can include a statement in an assertion that limits it to one intended service provider (e.g., "this document authenticates user FOO for Google service BAR"). Users of this library will simply see "this document authenticates user FOO" and authenticate that user, even though this document was never intended for them. Again it's been awhile so I might be mischaracterizing it a bit but that should be the gist.
janci 5y ago
Isn't this the usual practice with any nontrivial XML handling? You get some WSDL and XSDs of a coarse-grained service (as in having 300 optional input elements with no idea what can be combined or not) so you throw real-world examples on your implementation until it works.
- stouset 5y ago
  "Throw stuff at it until it looks like it works" is probably not the development practice you want to employ for an authentication service.
  - janci 5y ago
    Surely not! I'm just saying I'm not surprised too much.
    - jrockway 5y ago
      I think there's a mismatch between security and software engineering. Software engineers are paid to make things work, but the point of security is to make things not work (for attackers). So there is that fundamental mismatch which leads to a lot of strife, I think. (The real answer, of course, is that the software needs to do two things instead of just one. It needs to permit authorized users AND deny unauthorized users. One is not more important than the other, both must work perfectly. But when testing it, you're mostly happy when you, the authorized user, are allowed to log in, and so tend to forget about the "what if the assertion is malformed" case.) I think it's funny because engineers are always portrayed as being grumpy and that they hate everything, but if you look at how they handle errors or security, they seem like the type of people that like everything and always imagine the happy path ;)
      - 1cvmask 5y ago
        You framed the issue perfectly. A mismatch of competing incentives. That seems to be a problem everywhere beyond just security. it seems that bonuses should be tied to security as well. Then maybe software engineers would be incentivized to build things securely. A clawback of bonuses for security vulnerabilities might be an interesting idea? Or better yet bigger bonuses if there are no breaches.
jcranmer 5y ago
I get the feeling that this seems to be a common issue with any authentication or authorization "standard" nowadays. When I had to implement SASL OUATHBEARER support, and I started poking around how to actually implement OAuth2 from the standard itself, it was similarly full of "here's several ways that you might do something, and whether or not they're supported by the provider is implementation-defined, and what you have to provide with the requests is implementation-defined, and where you go to find the stuff is implementation-defined." And frustratingly for me--trying to implement this in the course of a SASL method as a client--there's not even anything hooked up in SASL that might have hinted at the client what to do. Which is insane because the entire point of SASL is to bridge the gap of "how to request authorization given username, hostname, protocol." It makes me want to retreat back into Kerberos as a better way of supporting SSO than anything invented in the decades after it.
- icedchai 5y ago
  Last I checked, and having worked on a few oauth clients, OAuth2 is more of a set of rough guidelines.
  - sarnowski 5y ago
    Correct, it’s officially a framework and not a protocol. It’s a framework to build a specific protocol which then is using the same patterns as other OAuth2 based protocols but not necessarily compatible. For example URL endpoints are not defined in a strict sense and the provider can also add arbitrary parameters to calls as long as the basic OAuth2 parameter are present as well. OpenID Connect 1.0 builds on that to make the framework more strict. RFC6749 The OAuth 2.0 Authorization _Framework_
aidenn0 5y ago
I ran into a similar issue trying to write an FTP client implementation from the spec. It turns out that most servers don't support 90% of the spec, and some very important things (such as the output format of LS) are completely unspecified. I threw out that first implementation and just got the 3 most popular servers and made it work with them. Random example: I'd like to present directories and files differently so users can chdir to a directory. There was no single method for detecting a symlink to a directory that worked on any 2 of the 3 servers I targeted for support (other than issuing a CHDIR and checking for success, and I wasn't going to issue a CHDIR for every item found when listing a directory for performance reasons; I did have to issue a CHDIR for every symlink on one of the 3 though).
- cmckn 5y ago
  I was once tasked with writing a crawler for FTP servers and lost about half the hair on my head over this madness. Just when I tuned the robot to work with 10 servers, an 11th would pop up that behaved totally differently. Thankfully, I'm not in that job anymore -- I'll probably never touch FTP again.
- klysm 5y ago
  The arbitrary parameter stuff can be a real pain because there’s some loose conventions around which parameters to use, but sometimes they’re named slightly differently or clash. Terribly annoying to work with.
bawolff 5y ago
XMLSignature is one of the worst security specs i have ever read. If you want to learn what not to do, just read that.
rkeene2 5y ago
I implemented a SAML IdP from the spec and it wasn't too difficult to get through (I implemented everything including the XML parser, XML DS (Digital Signature), XML C14N (Canonicalization), RSA implementation). I did take some shortcuts in the IdP part which just needs to produce but not validate XML DS (unless you're supporting signed requests).
- rkeene2 5y ago
  URL for it: https://github.com/rkeene/saml-idp https://github.com/rkeene/saml-idp
  - stouset 5y ago
    This is actually something I hadn’t really considered before. Writing an IdP is in many ways probably much easier than writing a service provider because there are many fewer knobs you need to turn. Not none, but fewer. You probably have one template for assertions that contains all the clauses and conditions you care about and you set them all every time. Clients can be much harder because you can any number of combinations of instructions thrown your way and you have to respect their intent. This is much harder to deal with.
robbedpeter 5y ago
Would something like this be a good use case for copilot / gpt based coding? You have massive specs in relatively structured documents, which could be used in isolated parses as prompts. The particular tuning of the model shouldn't matter if the source specifications and good examples are in the training data already. Use few shot prompts with well put together examples, then iterating over every item in the specs, and finally running the generated code through a human review process - seems like you could shift a lot of effort towards validation, and in doubt so, devise a process subject to further automation. Crawling the resulting codebase with various prompts like "this part of the specification could be improved by..." and then review a set of 20 completions for any real value. This might work for xml,html, and so forth, but even if you only have a system that generates a baseline library for one standard, it could be powerful.
- stouset 5y ago
  Comprehension of the details of a security standard is necessary for semantically correct and secure implementation of that security standard. I can't personally fathom how throwing GPT at the problem will improve things, but I can trivially see how it could result in code that looks correct to the author but isn't to anyone who read the spec. Maybe that's a failure of imagination on my part. Then again, I suppose that's the situation we're already in where well-meaning authors don't think to actually read the specs and so don't actually comprehend any of the details.
- ak217 5y ago
  Most problems with security specs and libraries that implement them are communication problems. They involve people incompletely describing or understanding their requirements, capabilities, or threat model. Often this also involves providing/using interfaces that are not ergonomic (https://github.com/google/mundane/blob/master/DESIGN.md https://github.com/google/mundane/blob/master/DESIGN.md), which in turn can be caused by the spec trying to do too much (as XML Signature does). I don't know how GPT could help with that. If anything I would expect it to bias toward things it has already seen, which is the opposite of what you want when writing a new spec/library aiming to avoid past mistakes.
- jcranmer 5y ago
hasa 5y ago
I feel that all authentication standards are bloated. Maybe there are reasons why they are like that.
thayne 5y ago
On the flip side, I've at least one SAML implementation that was written based on the spec, that then failed in the wild because identity providers didn't follow the spec. Go figure.
cratermoon 5y ago
The decryption and signature verification is the worst nightmare. Why? Because the spec signs the actual bytes of the XML, which means that if you change the order of two elements, even if everything else is exactly identical, the signature changes. Everything has to preserve the XML exactly as generated, down to the character set, or else the verification breaks. Edit: I know the article talks about this, but my point is that his description of the round-trip instability isn't quite right. You can't change the bytes arbitrarily.
commandlinefan 5y ago
> The standard is enormous And somehow still manages not to include any helpful description of SAML.

lol768 5y ago

> On top of this, I have inside knowledge that some extremely common libraries that implement SAML were built not by reading and understanding the spec, but simply by looking at sample XML documents in the wild and writing code that handled it. Yup, and its complexity leads to something of a shortage of libraries in the first place so there's not much choice. There's still no good modern maintained OSS .NET 5 library for doing SAML. There are some commercial offerings (ComponentSpace have a neat product and they weren't affected by VU#475445 either) but my worry is that folks will end up using whatever free libraries they can find without thinking too much about the consequences.

8mcoLwmQoWZ3s 5y ago
By "doing SAML" do you mean from the server perspective or the client perspective? This simple client library works with .NET 5. https://github.com/jitbit/AspNetSaml/ https://github.com/jitbit/AspNetSaml/
- mwcampbell 5y ago
  If I'm not mistaken, the correct terms are identity provider and service provider, respectively. After all, they're both servers, in the sense that they both receive requests from the other.
  - checker 5y ago
    The coolest part imo is that the the servers aren't communicating directly to one another. The client brokers the communication so that both IDP and SP can be on completely different networks (bridged by the client). You probably already knew this but sharing because I find it interesting (and for the edification of those that are unfamiliar).
- stouset 5y ago
  This client library is nowhere near enough by itself. I could tell this within five seconds, as it's a single source file only a few hundred lines long. It checks that you get a signed assertion back, which is valid, and lets you pull out attributes. That's great, but there are a gajillion tags that can be set within an assertion that have genuine, important security implications and zero of them are handled by this library. If your IdP sends those tags along with the assertion, this library will happily ignore them unless you implement support yourself. And I would bet heavily that approximating zero of this library's users write that support. One trivial example, there's a `<saml:Conditions>` tag. This tag can have all sorts of attributes like `NotBefore` or `NotOnOrAfter` that specify a time period for which the assertion is valid. This library does not do anything to implement support for `<saml:Conditions>` tags at all, so every consumer of this library will happily process assertions as valid even if they're outside of the duration for which the IdP is claiming it's valid. If I get my hands on an assertion at some point in time, I can hold onto it and replay it forever to unsuspecting users of this library. There are other attributes that can set conditions, and there are other tags with implications on whether or not an assertion should be considered valid. None of them are handled. Further, only minimal document validation is performed. The signature is checked, but the Issuer is not. Multiple Issuers can share a signing key, though I suspect actual cases of this are rare. At any rate, there's a billion ways users of this library can blithely chug along with an assertion that isn't valid and shouldn't be trusted, but nobody notices or cares because the happy path works. With authentication, yes it's important that legitimate users are able to sign in. But it's as important that illegitimate authentication attempts are disallowed. And this half is completely forgotten about in literally 100% of the common SAML service provider libraries I've seen in the wild (which is admittedly not 100% of the ones available, just the ones I've looked at). Edit: As @tptacek points out, every user of this library is trivially susceptible to AudienceRestriction authz vulnerabilities. An IdP can include a statement in an assertion that limits it to one intended service provider (e.g., "this document authenticates user FOO for Google service BAR"). Users of this library will simply see "this document authenticates user FOO" and authenticate that user, even though this document was never intended for them. Again it's been awhile so I might be mischaracterizing it a bit but that should be the gist.

janci 5y ago

Isn't this the usual practice with any nontrivial XML handling? You get some WSDL and XSDs of a coarse-grained service (as in having 300 optional input elements with no idea what can be combined or not) so you throw real-world examples on your implementation until it works.

stouset 5y ago
"Throw stuff at it until it looks like it works" is probably not the development practice you want to employ for an authentication service.
- janci 5y ago
  Surely not! I'm just saying I'm not surprised too much.
  - jrockway 5y ago
    I think there's a mismatch between security and software engineering. Software engineers are paid to make things work, but the point of security is to make things not work (for attackers). So there is that fundamental mismatch which leads to a lot of strife, I think. (The real answer, of course, is that the software needs to do two things instead of just one. It needs to permit authorized users AND deny unauthorized users. One is not more important than the other, both must work perfectly. But when testing it, you're mostly happy when you, the authorized user, are allowed to log in, and so tend to forget about the "what if the assertion is malformed" case.) I think it's funny because engineers are always portrayed as being grumpy and that they hate everything, but if you look at how they handle errors or security, they seem like the type of people that like everything and always imagine the happy path ;)
    - 1cvmask 5y ago
      You framed the issue perfectly. A mismatch of competing incentives. That seems to be a problem everywhere beyond just security. it seems that bonuses should be tied to security as well. Then maybe software engineers would be incentivized to build things securely. A clawback of bonuses for security vulnerabilities might be an interesting idea? Or better yet bigger bonuses if there are no breaches.

jcranmer 5y ago

I get the feeling that this seems to be a common issue with any authentication or authorization "standard" nowadays. When I had to implement SASL OUATHBEARER support, and I started poking around how to actually implement OAuth2 from the standard itself, it was similarly full of "here's several ways that you might do something, and whether or not they're supported by the provider is implementation-defined, and what you have to provide with the requests is implementation-defined, and where you go to find the stuff is implementation-defined." And frustratingly for me--trying to implement this in the course of a SASL method as a client--there's not even anything hooked up in SASL that might have hinted at the client what to do. Which is insane because the entire point of SASL is to bridge the gap of "how to request authorization given username, hostname, protocol." It makes me want to retreat back into Kerberos as a better way of supporting SSO than anything invented in the decades after it.

icedchai 5y ago
Last I checked, and having worked on a few oauth clients, OAuth2 is more of a set of rough guidelines.
- sarnowski 5y ago
  Correct, it’s officially a framework and not a protocol. It’s a framework to build a specific protocol which then is using the same patterns as other OAuth2 based protocols but not necessarily compatible. For example URL endpoints are not defined in a strict sense and the provider can also add arbitrary parameters to calls as long as the basic OAuth2 parameter are present as well. OpenID Connect 1.0 builds on that to make the framework more strict. RFC6749 The OAuth 2.0 Authorization _Framework_

aidenn0 5y ago

I ran into a similar issue trying to write an FTP client implementation from the spec. It turns out that most servers don't support 90% of the spec, and some very important things (such as the output format of LS) are completely unspecified. I threw out that first implementation and just got the 3 most popular servers and made it work with them. Random example: I'd like to present directories and files differently so users can chdir to a directory. There was no single method for detecting a symlink to a directory that worked on any 2 of the 3 servers I targeted for support (other than issuing a CHDIR and checking for success, and I wasn't going to issue a CHDIR for every item found when listing a directory for performance reasons; I did have to issue a CHDIR for every symlink on one of the 3 though).

cmckn 5y ago
I was once tasked with writing a crawler for FTP servers and lost about half the hair on my head over this madness. Just when I tuned the robot to work with 10 servers, an 11th would pop up that behaved totally differently. Thankfully, I'm not in that job anymore -- I'll probably never touch FTP again.
klysm 5y ago
The arbitrary parameter stuff can be a real pain because there’s some loose conventions around which parameters to use, but sometimes they’re named slightly differently or clash. Terribly annoying to work with.

bawolff 5y ago

XMLSignature is one of the worst security specs i have ever read. If you want to learn what not to do, just read that.

rkeene2 5y ago

I implemented a SAML IdP from the spec and it wasn't too difficult to get through (I implemented everything including the XML parser, XML DS (Digital Signature), XML C14N (Canonicalization), RSA implementation). I did take some shortcuts in the IdP part which just needs to produce but not validate XML DS (unless you're supporting signed requests).

rkeene2 5y ago
URL for it: https://github.com/rkeene/saml-idp https://github.com/rkeene/saml-idp
- stouset 5y ago
  This is actually something I hadn’t really considered before. Writing an IdP is in many ways probably much easier than writing a service provider because there are many fewer knobs you need to turn. Not none, but fewer. You probably have one template for assertions that contains all the clauses and conditions you care about and you set them all every time. Clients can be much harder because you can any number of combinations of instructions thrown your way and you have to respect their intent. This is much harder to deal with.

robbedpeter 5y ago

Would something like this be a good use case for copilot / gpt based coding? You have massive specs in relatively structured documents, which could be used in isolated parses as prompts. The particular tuning of the model shouldn't matter if the source specifications and good examples are in the training data already. Use few shot prompts with well put together examples, then iterating over every item in the specs, and finally running the generated code through a human review process - seems like you could shift a lot of effort towards validation, and in doubt so, devise a process subject to further automation. Crawling the resulting codebase with various prompts like "this part of the specification could be improved by..." and then review a set of 20 completions for any real value. This might work for xml,html, and so forth, but even if you only have a system that generates a baseline library for one standard, it could be powerful.

stouset 5y ago
Comprehension of the details of a security standard is necessary for semantically correct and secure implementation of that security standard. I can't personally fathom how throwing GPT at the problem will improve things, but I can trivially see how it could result in code that looks correct to the author but isn't to anyone who read the spec. Maybe that's a failure of imagination on my part. Then again, I suppose that's the situation we're already in where well-meaning authors don't think to actually read the specs and so don't actually comprehend any of the details.
ak217 5y ago
Most problems with security specs and libraries that implement them are communication problems. They involve people incompletely describing or understanding their requirements, capabilities, or threat model. Often this also involves providing/using interfaces that are not ergonomic (https://github.com/google/mundane/blob/master/DESIGN.md https://github.com/google/mundane/blob/master/DESIGN.md), which in turn can be caused by the spec trying to do too much (as XML Signature does). I don't know how GPT could help with that. If anything I would expect it to bias toward things it has already seen, which is the opposite of what you want when writing a new spec/library aiming to avoid past mistakes.
jcranmer 5y ago

hasa 5y ago

I feel that all authentication standards are bloated. Maybe there are reasons why they are like that.

thayne 5y ago

On the flip side, I've at least one SAML implementation that was written based on the spec, that then failed in the wild because identity providers didn't follow the spec. Go figure.

cratermoon 5y ago

The decryption and signature verification is the worst nightmare. Why? Because the spec signs the actual bytes of the XML, which means that if you change the order of two elements, even if everything else is exactly identical, the signature changes. Everything has to preserve the XML exactly as generated, down to the character set, or else the verification breaks. Edit: I know the article talks about this, but my point is that his description of the round-trip instability isn't quite right. You can't change the bytes arbitrarily.

commandlinefan 5y ago

> The standard is enormous And somehow still manages not to include any helpful description of SAML.