8 ms·

Using type inference to make web templates robust against XSS

43 points by revorad 14y ago

ufo 14y ago
I don't get why textual templates are so popular for generating HTML. Not only do they tend to have weird rules regarding whitespace and indentation but they also have no information about context when doing escaping (for example, just escaping special characters will not protect you if you add user content to an attribute and forget to add quotes since its easy to break out of the attribute with whitespace) data-name=${foo} data-name=Hello onmouseover=evilJavascript Using a document-oriented templating language avoids most of these problems without needing to do HTML parsing to check things (and HTML parsing is complicated and full of corner cases - its much worse than XML). (The article says that this not expressive enough since you can't open a tag in a subtemplate and close it in another one but I think you can see that as a feature...)
- mwcampbell 14y ago
  I agree with you. See also: http://glyph.twistedmatrix.com/2008/06/data-in-garbage-out.html http://glyph.twistedmatrix.com/2008/06/data-in-garbage-out.h...
- pekk 14y ago
  You don't have to write XML by hand. You can mostly use tools designed for working with HTML. The translation into final output is relatively simple. If you are handed HTML it is trivial to put in placeholders or slice it up rather than having to build it up from scratch in some alternate language.
  - ufo 14y ago
    I was saying that parsing HTML is harder than parsing XML, because of all the tricky corner cases with "malformed" stuff. Generating either of them is relatively easy to do. I don't undestand your point about placeholders though. Isn't the whole problem here that its hard to know how to escape the things you put into the placeholdes?
- gingerlime 14y ago
  can you please explain what you mean by 'document-oriented templating language'?
  - ufo 14y ago
    Basically anything that doesn't have "raw text" as the underlying model and that actually understands that its dealing with HTML. I am a big fan of embedded DSLs so I prefer when I libraries that let you freely mix HTML generation with code, but in a structured way so we don't generate invalid or unescaped markup (things like PHP fall in this unstructured category). An example of the embeded DSL approach is Haskell's Blaze[1]. Perl's Template::Declare[2] and Ruby's Markaby[3] seem to use similar approaches but I never worked with them myself. [1] http://www.jaspervdj.be/blaze/tutorial.html http://www.jaspervdj.be/blaze/tutorial.html [2] http://search.cpan.org/dist/Template-Declare/lib/Template/Declare.pm#SYNOPSIS http://search.cpan.org/dist/Template-Declare/lib/Template/De... [3] http://markaby.rubyforge.org/ http://markaby.rubyforge.org/ There are also more "templaty" alternatives that don't let you use the full power of an embeded DSL but that sitll assume that you are generating a document (instead of raw text) there are things like Hamlet[4]. I am sure there are also other similar examples fore mainstream languages but as I said, I like the embeded DSL stuff better... [4]http://hackage.haskell.org/package/hamlet http://hackage.haskell.org/package/hamlet
    - gingerlime 14y ago
      Thanks. Some interesting links and an interesting approach. Having looked (very briefly) into markaby, as far as I understand, the problem is still there to some extent though: "When you call helper methods inside Markaby, the output from those methods will be output to the stream. This is incredibly handy, since most Rails helpers output HTML tags."
    - masklinn 14y ago
      > I am sure there are also other similar examples fore mainstream languages For Python, there's Genshi: http://genshi.edgewall.org/ http://genshi.edgewall.org/
- sorbits 14y ago
  > I don't get why textual templates are so popular […] you can't open a tag in a subtemplate and close it in another one but I think you can see that as a feature For many, it’s frustrating having to work with a model where you can’t split up your template into multiple parts, without needing to make each part a valid document on its own. E.g. a blog would have templates for the page header/footer and article header/footer. It’s trivial to output the page header template, then iterate over the articles and output the article header/footer around the article text, which may be piped through various filters, and finally the page footer template. I think we can all imagine how such system looks when the templates are “raw text” (perhaps with some special tags for placeholders). I am interested in seeing how it will look if we are to use a document-oriented template language, as I certainly do like the idea of it, but I think in practice it adds overhead and restrictions, which doesn’t make the extra safety worth it.
  - nightpool 14y ago
    Why not just use a sub-template model then? Where you have a "page" template that includes the "header", "content" and "footer" template, possibly even decided at runtime?
    - sorbits 14y ago
      Such solution require much of the page-building logic (loops, conditions) to be moved into the template language, and the state transfer that goes with it. This means trivial tasks require editing multiple files, e.g. say we introduce pagination: the master template now need to be told by the controller, if it should include the “page n of m” template. I assume ufo avoids this problem (and the limited capabilities of template languages wrt. logic) by instead using host language libraries to generate the HTML, but IMO that has the downside of limited interoperability, as the templates are then just code written for some library.
  - ufo 14y ago
    If you really want to be able to split the document like that you can use a more "document streaming model", SAX-style, with separate functions for opening and closing the tags. What really matters is that the system knows what tags and attributes are so if can do things like quote attributes for you, check that closing tags match correctly and so on. (saying forcing complete subtemplates is a feature might have been too strong of a language, but I don't think its that big of a deal since you can usually work around it by splitting things up a bit) As you said, the real advantage of text-based systems is that they can generate text in other languages (sql, configuration files, etc) and that its easier to interoperate with other tools that spit out raw output. However, I fell things get really kludgy when you try to add automatic escaping to the text-based systems, for the reasons outlined in the original article.
cdoxsey 14y ago
Go's HTML templates implement similar escaping based on context: (http://golang.org/pkg/html/template/ http://golang.org/pkg/html/template/) "This package understands HTML, CSS, JavaScript, and URIs. It adds sanitizing functions to each simple action pipeline, so given the excerpt <a href="/search?q={{.}}">{{.}}</a> At parse time each {{.}} is overwritten to add escaping functions as necessary. In this case it becomes <a href="/search?q={{. | urlquery}}">{{. | html}}</a>"
- ufo 14y ago
  Do you know what approach the go template package actually uses to do the escaping? The original article is pretty comprehensive as to what alternatives people tend to use and lists pros and cons for most of them. However, when I searched to source code I found a link[1] back to the original article so now I'm just more confused! [1] http://golang.org/src/pkg/html/template/doc.go#L169 http://golang.org/src/pkg/html/template/doc.go#L169
  - carbocation 14y ago
    Here is escape.go from html/template : https://code.google.com/p/go/source/browse/src/pkg/html/template/escape.go?name=release-branch.go1 https://code.google.com/p/go/source/browse/src/pkg/html/temp...
    - ufo 14y ago
      Wait a minute. If its just adding escaping functions to the templates, who is actually running the templates?
CGamesPlay 14y ago
The example in the solution sketch leads me to believe the author is misdirected. URLs have different attack vectors than HTML and have different escaping rules. In the solution sketch given, the HTML might be safe, but I can easily put any parameters I want into the generated URL. When designing systems like this, each security boundary should be handled separately. Create a URL class that supports escaping fields safely. Create an HTML template engine that supports escaping contents safely. URL.toString will give a guaranteed-safe URL, which can be inserted into a guaranteed-safe attribute later on.
- ufo 14y ago
  My impression was that they were trying to go for a more "middle ground" approach to things that made it easy to convert old existing templates (written in less safe dialects like jQuery templates). Since the legacy code allows you to have inline URLs that you put things in you can't force the users to refactor things to generate all the URLs outside the templates.

43 points by revorad 14y ago

ufo 14y ago

I don't get why textual templates are so popular for generating HTML. Not only do they tend to have weird rules regarding whitespace and indentation but they also have no information about context when doing escaping (for example, just escaping special characters will not protect you if you add user content to an attribute and forget to add quotes since its easy to break out of the attribute with whitespace) data-name=${foo} data-name=Hello onmouseover=evilJavascript Using a document-oriented templating language avoids most of these problems without needing to do HTML parsing to check things (and HTML parsing is complicated and full of corner cases - its much worse than XML). (The article says that this not expressive enough since you can't open a tag in a subtemplate and close it in another one but I think you can see that as a feature...)

mwcampbell 14y ago
I agree with you. See also: http://glyph.twistedmatrix.com/2008/06/data-in-garbage-out.html http://glyph.twistedmatrix.com/2008/06/data-in-garbage-out.h...
pekk 14y ago
You don't have to write XML by hand. You can mostly use tools designed for working with HTML. The translation into final output is relatively simple. If you are handed HTML it is trivial to put in placeholders or slice it up rather than having to build it up from scratch in some alternate language.
- ufo 14y ago
  I was saying that parsing HTML is harder than parsing XML, because of all the tricky corner cases with "malformed" stuff. Generating either of them is relatively easy to do. I don't undestand your point about placeholders though. Isn't the whole problem here that its hard to know how to escape the things you put into the placeholdes?
gingerlime 14y ago
can you please explain what you mean by 'document-oriented templating language'?
- ufo 14y ago
  Basically anything that doesn't have "raw text" as the underlying model and that actually understands that its dealing with HTML. I am a big fan of embedded DSLs so I prefer when I libraries that let you freely mix HTML generation with code, but in a structured way so we don't generate invalid or unescaped markup (things like PHP fall in this unstructured category). An example of the embeded DSL approach is Haskell's Blaze[1]. Perl's Template::Declare[2] and Ruby's Markaby[3] seem to use similar approaches but I never worked with them myself. [1] http://www.jaspervdj.be/blaze/tutorial.html http://www.jaspervdj.be/blaze/tutorial.html [2] http://search.cpan.org/dist/Template-Declare/lib/Template/Declare.pm#SYNOPSIS http://search.cpan.org/dist/Template-Declare/lib/Template/De... [3] http://markaby.rubyforge.org/ http://markaby.rubyforge.org/ There are also more "templaty" alternatives that don't let you use the full power of an embeded DSL but that sitll assume that you are generating a document (instead of raw text) there are things like Hamlet[4]. I am sure there are also other similar examples fore mainstream languages but as I said, I like the embeded DSL stuff better... [4]http://hackage.haskell.org/package/hamlet http://hackage.haskell.org/package/hamlet
  - gingerlime 14y ago
    Thanks. Some interesting links and an interesting approach. Having looked (very briefly) into markaby, as far as I understand, the problem is still there to some extent though: "When you call helper methods inside Markaby, the output from those methods will be output to the stream. This is incredibly handy, since most Rails helpers output HTML tags."
  - masklinn 14y ago
    > I am sure there are also other similar examples fore mainstream languages For Python, there's Genshi: http://genshi.edgewall.org/ http://genshi.edgewall.org/
sorbits 14y ago
> I don't get why textual templates are so popular […] you can't open a tag in a subtemplate and close it in another one but I think you can see that as a feature For many, it’s frustrating having to work with a model where you can’t split up your template into multiple parts, without needing to make each part a valid document on its own. E.g. a blog would have templates for the page header/footer and article header/footer. It’s trivial to output the page header template, then iterate over the articles and output the article header/footer around the article text, which may be piped through various filters, and finally the page footer template. I think we can all imagine how such system looks when the templates are “raw text” (perhaps with some special tags for placeholders). I am interested in seeing how it will look if we are to use a document-oriented template language, as I certainly do like the idea of it, but I think in practice it adds overhead and restrictions, which doesn’t make the extra safety worth it.
- nightpool 14y ago
  Why not just use a sub-template model then? Where you have a "page" template that includes the "header", "content" and "footer" template, possibly even decided at runtime?
  - sorbits 14y ago
    Such solution require much of the page-building logic (loops, conditions) to be moved into the template language, and the state transfer that goes with it. This means trivial tasks require editing multiple files, e.g. say we introduce pagination: the master template now need to be told by the controller, if it should include the “page n of m” template. I assume ufo avoids this problem (and the limited capabilities of template languages wrt. logic) by instead using host language libraries to generate the HTML, but IMO that has the downside of limited interoperability, as the templates are then just code written for some library.
- ufo 14y ago
  If you really want to be able to split the document like that you can use a more "document streaming model", SAX-style, with separate functions for opening and closing the tags. What really matters is that the system knows what tags and attributes are so if can do things like quote attributes for you, check that closing tags match correctly and so on. (saying forcing complete subtemplates is a feature might have been too strong of a language, but I don't think its that big of a deal since you can usually work around it by splitting things up a bit) As you said, the real advantage of text-based systems is that they can generate text in other languages (sql, configuration files, etc) and that its easier to interoperate with other tools that spit out raw output. However, I fell things get really kludgy when you try to add automatic escaping to the text-based systems, for the reasons outlined in the original article.

cdoxsey 14y ago

Go's HTML templates implement similar escaping based on context: (http://golang.org/pkg/html/template/ http://golang.org/pkg/html/template/) "This package understands HTML, CSS, JavaScript, and URIs. It adds sanitizing functions to each simple action pipeline, so given the excerpt <a href="/search?q={{.}}">{{.}}</a> At parse time each {{.}} is overwritten to add escaping functions as necessary. In this case it becomes <a href="/search?q={{. | urlquery}}">{{. | html}}</a>"

ufo 14y ago
Do you know what approach the go template package actually uses to do the escaping? The original article is pretty comprehensive as to what alternatives people tend to use and lists pros and cons for most of them. However, when I searched to source code I found a link[1] back to the original article so now I'm just more confused! [1] http://golang.org/src/pkg/html/template/doc.go#L169 http://golang.org/src/pkg/html/template/doc.go#L169
- carbocation 14y ago
  Here is escape.go from html/template : https://code.google.com/p/go/source/browse/src/pkg/html/template/escape.go?name=release-branch.go1 https://code.google.com/p/go/source/browse/src/pkg/html/temp...
  - ufo 14y ago
    Wait a minute. If its just adding escaping functions to the templates, who is actually running the templates?

CGamesPlay 14y ago

The example in the solution sketch leads me to believe the author is misdirected. URLs have different attack vectors than HTML and have different escaping rules. In the solution sketch given, the HTML might be safe, but I can easily put any parameters I want into the generated URL. When designing systems like this, each security boundary should be handled separately. Create a URL class that supports escaping fields safely. Create an HTML template engine that supports escaping contents safely. URL.toString will give a guaranteed-safe URL, which can be inserted into a guaranteed-safe attribute later on.

ufo 14y ago
My impression was that they were trying to go for a more "middle ground" approach to things that made it easy to convert old existing templates (written in less safe dialects like jQuery templates). Since the legacy code allows you to have inline URLs that you put things in you can't force the users to refactor things to generate all the URLs outside the templates.