4 ms·

"Just by replacing the escaping function with a more optimized one, we've reduced the average request time by 45ms, and we're allocating 20,000 less Ruby object

by minimax 14y ago

"Just by replacing the escaping function with a more optimized one, we've reduced the average request time by 45ms, and we're allocating 20,000 less Ruby objects per request. That was a lot of escaped HTML right there!" I generally stay away from web development so forgive me if this one is obvious, but why does so much text need to have HTML escaping performed in order to render the page? Also is there a way to quantify how much text that is? Like a few K per page or a few hundred K?

deleted 14y ago
[deleted]
Cyranix 14y ago
`phillmv has identified the biggest concern for sure, but more broadly speaking, you have issues whenever you cross one of two boundaries: 1) escaping context [characters get escaped differently in URLs, JavaScript, HTML, etc.] and 2) user-defined content. The latter is the source of XSS attacks, but the former is a pain too. Let's say that you run a stock photo website, and your database has images and captions. You decide you want to use captions for the alt attribute of your images, and one of your captions is Cat saying "meow!". If you blindly insert that caption into your HTML without escaping, you could end up with <img src="cat.jpg" alt="Cat saying "meow!""> And now you've got crappy HTML on your hands. It's the same principle as XSS attacks, but you do it to yourself with less disastrous (but still unwanted) effects. EDIT: Hm, seems like the comment I referenced is now gone? `phillmv pointed out that cross-site scripting (XSS) is an important reason to use escaping appropriately (and often widely).
lobster_johnson 14y ago
The problem occurs with templates such as in this (HAML example): - posts.each do |post| %li = link_to post.title, post_url(post), class: "post_link" Note that in the case of Rails 2.3 (which the article describes), Rails doesn't know if post.title, post_url() or "post_link" have been escaped or not. It originates at a time when the Rails people were fairly lax and ignorant about encodings and sanitization in general. For the post.title bit, the link_to method just passes it along, assuming it's valid HTML. Which means it's an injection point if the post was user-submitted. The URL and the {class: "post_link"} are passed through escape_once, described in the article. As you can see, the amount of data that must potentially be sanitized/encoded can be a lot; lots of small pieces of text that add up to a lot of overhead. Rails 3.x fixes this by assuming that no strings contain sanitized HTML; therefore, when you insert data into a template, it will automatically escape everything. This is good. If you know that something is already sanitized, you can declare it as such: = link_to post.title.html_safe This flags the string as safe, and link_to will not need to escape it.
JangoSteve 14y ago
My guess is that it has something to do with the fact that Github is made up almost entirely of user generated content. Just load a Github project page. From top to bottom, you have: * Username in the header (which is probably validated with a specific format, so probably doesn't need escaping). * Project summary entered when the project is created. * Drop-down with list of branch and tag names. * Latest commit message. * List of names for all folders and files tracked in git for the project. * Readme text. That's just the homepage for a project, and each one of those things could potentially contain malicious HTML or JS, meaning it should be escaped before rendered on the page.
- theycallmemorty 14y ago
  Not to mention the pages that display actual code.

by minimax 14y ago

deleted 14y ago

[deleted]

Cyranix 14y ago

`phillmv has identified the biggest concern for sure, but more broadly speaking, you have issues whenever you cross one of two boundaries: 1) escaping context [characters get escaped differently in URLs, JavaScript, HTML, etc.] and 2) user-defined content. The latter is the source of XSS attacks, but the former is a pain too. Let's say that you run a stock photo website, and your database has images and captions. You decide you want to use captions for the alt attribute of your images, and one of your captions is Cat saying "meow!". If you blindly insert that caption into your HTML without escaping, you could end up with <img src="cat.jpg" alt="Cat saying "meow!""> And now you've got crappy HTML on your hands. It's the same principle as XSS attacks, but you do it to yourself with less disastrous (but still unwanted) effects. EDIT: Hm, seems like the comment I referenced is now gone? `phillmv pointed out that cross-site scripting (XSS) is an important reason to use escaping appropriately (and often widely).

lobster_johnson 14y ago

The problem occurs with templates such as in this (HAML example): - posts.each do |post| %li = link_to post.title, post_url(post), class: "post_link" Note that in the case of Rails 2.3 (which the article describes), Rails doesn't know if post.title, post_url() or "post_link" have been escaped or not. It originates at a time when the Rails people were fairly lax and ignorant about encodings and sanitization in general. For the post.title bit, the link_to method just passes it along, assuming it's valid HTML. Which means it's an injection point if the post was user-submitted. The URL and the {class: "post_link"} are passed through escape_once, described in the article. As you can see, the amount of data that must potentially be sanitized/encoded can be a lot; lots of small pieces of text that add up to a lot of overhead. Rails 3.x fixes this by assuming that no strings contain sanitized HTML; therefore, when you insert data into a template, it will automatically escape everything. This is good. If you know that something is already sanitized, you can declare it as such: = link_to post.title.html_safe This flags the string as safe, and link_to will not need to escape it.

JangoSteve 14y ago

My guess is that it has something to do with the fact that Github is made up almost entirely of user generated content. Just load a Github project page. From top to bottom, you have: * Username in the header (which is probably validated with a specific format, so probably doesn't need escaping). * Project summary entered when the project is created. * Drop-down with list of branch and tag names. * Latest commit message. * List of names for all folders and files tracked in git for the project. * Readme text. That's just the homepage for a project, and each one of those things could potentially contain malicious HTML or JS, meaning it should be escaped before rendered on the page.

theycallmemorty 14y ago
Not to mention the pages that display actual code.