3 ms·

There are 3 correct ways (and 1 incorrect way) to secure your strings in a web application: 1. Output escaping (server-side) 2. Output sanitization (server-si

There are 3 correct ways (and 1 incorrect way) to secure your strings in a web application: 1. Output escaping (server-side) 2. Output sanitization (server-side) 3. Output sanitization (client-side) - i.e. this standard 4. [Wrong] Input sanitization (server-side) (not to be confused with input validation, which is good but unrelated to this conversation) Without boring with the details of why 4 is wrong, normally you should choose either 1, 2 or 3 depending on different applications: (1) should generally be used in almost ALL cases. Contextual escaping is orders of magnitude more secure than sanitization, and is usable by any application not rendering user-provided rich text. Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well. (2) should only be used if you are rendering user-provided rich-text. These cases are quite rare, and chances are if you need this feature you have a large application with lots of experienced engineers and you have the resources to put in place appropriate caching mechanisms. (3) is the same as (2) but offloads the work to the client (no caching necessary) and has a resultant negative impact on the performance of your app (client-side UI latency). Making browsers fast is already a gargantuan challenge for browser-makers; adding client-sized sanitization to the mix will exacerbate this. But mainly, this new standard will encourage inexperienced engineers to use (3) where (1) is much more secure for most applications.

beepy 6y ago
I'll bite - why is 2 right but 4 wrong? Is it because it's not reversible? Is it in case your sanitization rules change?
- lucideer 6y ago
  Firstly, to distinguish sanitization -vs- validation: sanitization retains information lossily, validation throws. Validation may increase security, but the primary motivator is stability (avoiding unexpected state) and UX (relevant error messaging). Input sanitization is intended purely as a security measure but is at best insufficient and usually reduces system security. The first problem is that input sanitization is sanitizing without context: you don't know what threats you're fighting because threat vectors of this variety target the output format. Will your input variable be included into a HTML template, an Atom feed, saved to some file that's parsed later, sent in a JSON API, templated into a style or script block, used in an SQL query. The possibilities aren't known at input (or if they are they will change as your application grows) so protecting against all possibilities isn't viable. Often people html-escape or blindly html-strip their inputs, regardless of whether they'll ever be used in a html template. Given the above you might think input sanitization is at worst useless, inefficient, but not harmful to security, but that brings us to the follow-up problems: lossiness & unknown state. If you're doing input sanitization, you're not doing input validation (at least not properly). Input sanitization is about accepting lossy values (threats removed) when you receive unexpected input. That means you're accepting unexpected input, which leads to all the problems input validation is designed to protect against. Finally, there is of course double-escaping. As mentioned above, people often html-escape as part of input sanitization. This essentially disables your ability to do reliable secure output escaping because that leads to double escaped values in output (or horrible double-escape-reversal hacks in output templating code). Generally you want to be securely output/escaping everything, which means you want a system where you can rely on always receiving clean unescaped values into your output template. I have encountered too many systems using a good, secure html templating library with output escaping baked in by default, where devs had to disable output escaping explicitly because some inputs were already pre-escaped. In theory you can keep track of which vars need raw output and which vars don't, but that's an unmanageable mess in practice. And its impossible to automate enforcement.
  - sirsinsalot 6y ago
    Or in other terms: Store Raw, output safe means you can improve output safety over time. Storing safe locks you in to that mutation.
    - lucideer 6y ago
      That's a really nice succinct way of putting it. Will steal.
      - sirsinsalot 6y ago
        Just don't steal my weird capitalisation hah
IshKebab 6y ago
I think that's a bit missing the point of why sanitization is wrong (e.g. 2 & 3). The core problem is that blacklisting elements is really error prone. People frequently find ways around it. You really shouldn't do "block this element, allow this element" type sanitizing ever. What you should do is: 1. Parse the input into an AST that only allows safe nodes. I.e. the AST should have no `<script>` node at all. 2. Write it back out as HTML. It's more work.
- lucideer 6y ago
  > You really shouldn't do "block this element, allow this element" type sanitizing ever. > What you should do is: > 1. Parse the input into an AST that only allows safe nodes. I.e. the AST should have no <script> node at all. 2. Write it back out as HTML. Firstly I don't see the difference between what you say you shouldn't do and what you say you should do. This is what a sanitizer does: it parses the input into an AST that only allows safe things. It does "block this element, allow this element" on an AST. So I don't understand what things you're contrasting? Secondly, I guess this was just brevity, but disallowing script nodes is the tip of the iceberg when it comes to sanitization. You need to deal with a plethora of vectors involving scriptable & interactivity attributes, and other "active" non-script elements.
RcouF1uZ4gsC 6y ago
> Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well. Be very careful about that assumption for Markdown. Many markdown processors will pass html tags through I modified so you could have a script in Markdown.
- lucideer 6y ago
  Good point. Maybe I should've said BB :D
megous 6y ago
5. Use proper API instead of escaping. (client-side) Say you assign your non-static data to el.textContent instead of concatenating strings into something that's to be interpreted as markup/code. Or you use DOMParser to parse markup safely, and then extract data from it or prune it using a whitelist via regular DOM API. 5a. Block all access to unsafe ways to use strings as code/markup via trusted-types CSP. (at least during development) Conceptually the same thing you do with SQL server side. Parametrize the queries, instead of escaping and pasting into the query.
- lucideer 6y ago
  Great point! It wanted to edit the comment to change (1) to (server/client) but I passed my edit timeout. I would include your (5) within (1). `textContent` and other DOM methods like `setAttribute` are effectively secure output-escaping on the client. Your (5a) is an excellent extra measure. In this area, I'd also add security-focused linting for (1) and (5)–e.g. for (5), to ensure secure DOM methods are used, I use Mozilla's `eslint-plugin-no-unsanitized`[0] plugin for all my personal & work projects. [0] https://github.com/mozilla/eslint-plugin-no-unsanitized/ https://github.com/mozilla/eslint-plugin-no-unsanitized/
felixfbecker 6y ago
> Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well. Markdown does not side-step the need for sanitization. You can embed HTML inside markdown, so you actually need sanitization for exactly the use case of rendering markdown on the client.

beepy 6y ago

I'll bite - why is 2 right but 4 wrong? Is it because it's not reversible? Is it in case your sanitization rules change?

lucideer 6y ago
Firstly, to distinguish sanitization -vs- validation: sanitization retains information lossily, validation throws. Validation may increase security, but the primary motivator is stability (avoiding unexpected state) and UX (relevant error messaging). Input sanitization is intended purely as a security measure but is at best insufficient and usually reduces system security. The first problem is that input sanitization is sanitizing without context: you don't know what threats you're fighting because threat vectors of this variety target the output format. Will your input variable be included into a HTML template, an Atom feed, saved to some file that's parsed later, sent in a JSON API, templated into a style or script block, used in an SQL query. The possibilities aren't known at input (or if they are they will change as your application grows) so protecting against all possibilities isn't viable. Often people html-escape or blindly html-strip their inputs, regardless of whether they'll ever be used in a html template. Given the above you might think input sanitization is at worst useless, inefficient, but not harmful to security, but that brings us to the follow-up problems: lossiness & unknown state. If you're doing input sanitization, you're not doing input validation (at least not properly). Input sanitization is about accepting lossy values (threats removed) when you receive unexpected input. That means you're accepting unexpected input, which leads to all the problems input validation is designed to protect against. Finally, there is of course double-escaping. As mentioned above, people often html-escape as part of input sanitization. This essentially disables your ability to do reliable secure output escaping because that leads to double escaped values in output (or horrible double-escape-reversal hacks in output templating code). Generally you want to be securely output/escaping everything, which means you want a system where you can rely on always receiving clean unescaped values into your output template. I have encountered too many systems using a good, secure html templating library with output escaping baked in by default, where devs had to disable output escaping explicitly because some inputs were already pre-escaped. In theory you can keep track of which vars need raw output and which vars don't, but that's an unmanageable mess in practice. And its impossible to automate enforcement.
- sirsinsalot 6y ago
  Or in other terms: Store Raw, output safe means you can improve output safety over time. Storing safe locks you in to that mutation.
  - lucideer 6y ago
    That's a really nice succinct way of putting it. Will steal.
    - sirsinsalot 6y ago
      Just don't steal my weird capitalisation hah

IshKebab 6y ago

I think that's a bit missing the point of why sanitization is wrong (e.g. 2 & 3). The core problem is that blacklisting elements is really error prone. People frequently find ways around it. You really shouldn't do "block this element, allow this element" type sanitizing ever. What you should do is: 1. Parse the input into an AST that only allows safe nodes. I.e. the AST should have no `<script>` node at all. 2. Write it back out as HTML. It's more work.

lucideer 6y ago
> You really shouldn't do "block this element, allow this element" type sanitizing ever. > What you should do is: > 1. Parse the input into an AST that only allows safe nodes. I.e. the AST should have no <script> node at all. 2. Write it back out as HTML. Firstly I don't see the difference between what you say you shouldn't do and what you say you should do. This is what a sanitizer does: it parses the input into an AST that only allows safe things. It does "block this element, allow this element" on an AST. So I don't understand what things you're contrasting? Secondly, I guess this was just brevity, but disallowing script nodes is the tip of the iceberg when it comes to sanitization. You need to deal with a plethora of vectors involving scriptable & interactivity attributes, and other "active" non-script elements.

RcouF1uZ4gsC 6y ago

> Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well. Be very careful about that assumption for Markdown. Many markdown processors will pass html tags through I modified so you could have a script in Markdown.

lucideer 6y ago
Good point. Maybe I should've said BB :D

megous 6y ago

5. Use proper API instead of escaping. (client-side) Say you assign your non-static data to el.textContent instead of concatenating strings into something that's to be interpreted as markup/code. Or you use DOMParser to parse markup safely, and then extract data from it or prune it using a whitelist via regular DOM API. 5a. Block all access to unsafe ways to use strings as code/markup via trusted-types CSP. (at least during development) Conceptually the same thing you do with SQL server side. Parametrize the queries, instead of escaping and pasting into the query.

lucideer 6y ago
Great point! It wanted to edit the comment to change (1) to (server/client) but I passed my edit timeout. I would include your (5) within (1). `textContent` and other DOM methods like `setAttribute` are effectively secure output-escaping on the client. Your (5a) is an excellent extra measure. In this area, I'd also add security-focused linting for (1) and (5)–e.g. for (5), to ensure secure DOM methods are used, I use Mozilla's `eslint-plugin-no-unsanitized`[0] plugin for all my personal & work projects. [0] https://github.com/mozilla/eslint-plugin-no-unsanitized/ https://github.com/mozilla/eslint-plugin-no-unsanitized/

felixfbecker 6y ago

> Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well. Markdown does not side-step the need for sanitization. You can embed HTML inside markdown, so you actually need sanitization for exactly the use case of rendering markdown on the client.