3 ms·

The overall point is very true indeed, though I think it's not made particularly clear what the actual problem with sanitizing input is. The problem is that yo

The overall point is very true indeed, though I think it's not made particularly clear what the actual problem with sanitizing input is. The problem is that you are silently changing information, and that's an absolute no-go for reliable data processing, and the cause is that people think of, say, html, as "some kind of text/strings". HTML is a serialization of a tree, similarly, SQL is a serialization of a syntax tree ... - and if you want to add plain-text user input to such a serialized tree, you have to _convert_ it from, say, "plain text" to "HTML character data". You have to think of them as two different data types, and so when you want to use a value presented as one of the types as the other type, you don't have to "sanitize" it, even calling it "escaping" is confusing - you have to _convert_ it. And if it happens that some input can not be represented in the target type, then you have to _validate_ the input and _reject_ broken input.

chriswarbo 13y ago
Not sure why this is being downvoted. This "some kind of text/strings" approach is why we have "escaping" functions which turn strings into strings, instead of conversion functions from, say, "plain text" to "HTML character data". This prevents our computers from helping us, even if we're using an ivory tower type system with a whizz-bang IDE, since everything's just "String" so the compiler says OK. Here's an example of the alternative http://blog.moertel.com/posts/2006-10-18-a-type-based-solution-to-the-strings-problem.html http://blog.moertel.com/posts/2006-10-18-a-type-based-soluti... (remember that most of the code there is building the libraries; using them is simple and terse).
marcosdumay 13y ago
"Convert" is ambiguous. When you read the text "<head>" from a template file, you probably want to represent it as the HTML "<head>", but when you read it from the database, you may, or may not want to represent it as "<head>". People started using the word "sanitize" exactly because it conveys that information that "you want to treat it differently, depending of where it comes from". We also use the words "dirty" (sometimes "tainted") and "clean" conserving their usual relations to "sanitize". Now somebody wants throw away a very concise and expressive jargon just because some people are giving bad advice on the Internet?
- zAy0LfpBZLC8mAC 13y ago
  This has nothing to do with whether you read it from "a (template) file" or "the database", it's only about what _format_ it is in. If the template contains HTML, then the conversion to HTML (for the HTML part) is the identity function, of course, the same if the database contains HTML - if you want to use the same thing in a plain text email, you will have to convert to plain text. If, on the other hand, the template file or the database contains plain text, the reverse applies: conversion to plain text is the identity function, conversion to HTML is the usual replacement with entity references. That you are using "dirty/tainted" and "clean" only shows how deep the confusion is. There is some justification to use those terms when talking about before and after validation, but other than that it's probably an indication of confusion (which also seems to be the common usage). Take, for example, a general plain text field for optional free-form text. There is essentially nothing that could be validated (other than maybe that it's a valid UTF-8 string). Now, you want to generate a plain text email using the user input - how would you "sanitize" it? There is nothing "better"/"cleaner" about any particular encoding, be it plain text, HTML, SQL, or any other, they are simply different encodings, and you have to always use the correct one, not the "best one"/"cleanest one", and you have to always know what format the data that you are processing is in so that you can convert correctly. This jargon is not at all concise, actually (some people mean "remove 'strange' stuff/clean it", others mean "escape it", ...), and it makes you think in ways that obscure the actual problem that you are solving: Conversion between data types/data representations.

chriswarbo 13y ago

Not sure why this is being downvoted. This "some kind of text/strings" approach is why we have "escaping" functions which turn strings into strings, instead of conversion functions from, say, "plain text" to "HTML character data". This prevents our computers from helping us, even if we're using an ivory tower type system with a whizz-bang IDE, since everything's just "String" so the compiler says OK. Here's an example of the alternative http://blog.moertel.com/posts/2006-10-18-a-type-based-solution-to-the-strings-problem.html http://blog.moertel.com/posts/2006-10-18-a-type-based-soluti... (remember that most of the code there is building the libraries; using them is simple and terse).

marcosdumay 13y ago

"Convert" is ambiguous. When you read the text "<head>" from a template file, you probably want to represent it as the HTML "<head>", but when you read it from the database, you may, or may not want to represent it as "<head>". People started using the word "sanitize" exactly because it conveys that information that "you want to treat it differently, depending of where it comes from". We also use the words "dirty" (sometimes "tainted") and "clean" conserving their usual relations to "sanitize". Now somebody wants throw away a very concise and expressive jargon just because some people are giving bad advice on the Internet?

zAy0LfpBZLC8mAC 13y ago
This has nothing to do with whether you read it from "a (template) file" or "the database", it's only about what _format_ it is in. If the template contains HTML, then the conversion to HTML (for the HTML part) is the identity function, of course, the same if the database contains HTML - if you want to use the same thing in a plain text email, you will have to convert to plain text. If, on the other hand, the template file or the database contains plain text, the reverse applies: conversion to plain text is the identity function, conversion to HTML is the usual replacement with entity references. That you are using "dirty/tainted" and "clean" only shows how deep the confusion is. There is some justification to use those terms when talking about before and after validation, but other than that it's probably an indication of confusion (which also seems to be the common usage). Take, for example, a general plain text field for optional free-form text. There is essentially nothing that could be validated (other than maybe that it's a valid UTF-8 string). Now, you want to generate a plain text email using the user input - how would you "sanitize" it? There is nothing "better"/"cleaner" about any particular encoding, be it plain text, HTML, SQL, or any other, they are simply different encodings, and you have to always use the correct one, not the "best one"/"cleanest one", and you have to always know what format the data that you are processing is in so that you can convert correctly. This jargon is not at all concise, actually (some people mean "remove 'strange' stuff/clean it", others mean "escape it", ...), and it makes you think in ways that obscure the actual problem that you are solving: Conversion between data types/data representations.