4 ms·

Neat. Getting some good use out of HTML2Canvas as well. But always wondering why there is no way to get a bitmap from screen... something like window.getPixels(

by jstsch 12y ago

Neat. Getting some good use out of HTML2Canvas as well. But always wondering why there is no way to get a bitmap from screen... something like window.getPixels(x, y, width, height).

theallan 12y ago
Yes! This would open a huge number of possibilities. Simple things like a colour picker, to transitions based on the content of the window would be made possible. I would love to see this happen.
jevinskie 12y ago
There are privacy and security concerns to address. Similar to this situation: https://hacks.mozilla.org/2010/03/privacy-related-changes-coming-to-css-vistited/ https://hacks.mozilla.org/2010/03/privacy-related-changes-co...
- nawitus 12y ago
  One solution is for the browser to provide a "anonymized" bitmap picture, e.g. setting the links to default colors. I don't know how feasible that is. It would require a new rendering of the page, at least.
  - euank 12y ago
    That's not a solution for the general problem of getPixel + iframes though. Here, I'll give a simple example. Let's say there's a site that uses cookies to track logins. For example, hacker news. Now, hacker news does have the X-Frame-Option Deny, but let's assume it doesn't. So to figure out my hacker news username, all you have to do now is create an iframe with hacker news, and then getPixel on the area of the frame that contains usernames, run some trivial OCR, and done. Now, this issue is even more serious in other instances. For example, google talk widgets are embedded by iframes I believe. In both these cases, a fresh rendering would still have the inappropriate material due to cookies. If cookies aren't sent, e.g. you do a fresh render in a private tab, it would still have security concerns for anything that displays different, sometimes private, content based on ip address. For an example of that, you could render "private.internal.company.localsite" in an iframe and if a visitor from that company visited, even a cookie-less load would probably show private data due to the internal site relying on ip/nat controls.
    - comex 12y ago
      Any reason you couldn't just outright black out any pixels covered by different-origin iframes?
      - jgraham 12y ago
        Even some same-origin things, for example input type=file, which can reveal paths on the local filesystem, would have to be blanked out.
    - tlrobinson 12y ago
      There's no reason you couldn't apply the exact same "same origin policy" to elements (iframes, images, etc) when rendering into a bitmap.
      - euank 12y ago
        You could, but it would change scope a little. Right now, it's reasonable to expect that information will not be leaked to external servers, but user interactions can be faked. On an information only webpage (no user interactions) there's no need to have the X-Frame-Options to remain secure. If there's a way to access the data in that frame suddenly, that changes the necessity of that header.
        tlrobinson 12y ago
        I'm not sure what X-Frame-Options has to do with it. I'm suggesting either not allowing rendering of bitmaps containing different origin iframes/images, or blanking out those elements in the rendering. Canvas does exactly that by not allowing different origin images to be drawn into a canvas element.
- jstsch 12y ago
  Hmm, if I can get all underlying DOM, including CSS, what is the difference except massive convenience? I could imagine iframes of other origins to be blank, but that would be it, I think?
  - mbrubeck 12y ago
    Also cross-origin images, which would affect a large number of sites. And visited link coloring [1], although I supposed the browser could re-render the page without it for screenshot purposes. 1. http://dbaron.org/mozilla/visited-privacy http://dbaron.org/mozilla/visited-privacy
spankalee 12y ago
Firefox has had the element() CSS function for a while, but they're the only ones. https://developer.mozilla.org/en-US/docs/Web/CSS/element https://developer.mozilla.org/en-US/docs/Web/CSS/element I've written GUI builders that really could have used this for icons of views. Instead I had to render on the server. It'd be nice if it got more broadly adopted.

by jstsch 12y ago

Neat. Getting some good use out of HTML2Canvas as well. But always wondering why there is no way to get a bitmap from screen... something like window.getPixels(x, y, width, height).

theallan 12y ago

Yes! This would open a huge number of possibilities. Simple things like a colour picker, to transitions based on the content of the window would be made possible. I would love to see this happen.

jevinskie 12y ago

There are privacy and security concerns to address. Similar to this situation: https://hacks.mozilla.org/2010/03/privacy-related-changes-coming-to-css-vistited/ https://hacks.mozilla.org/2010/03/privacy-related-changes-co...

nawitus 12y ago
One solution is for the browser to provide a "anonymized" bitmap picture, e.g. setting the links to default colors. I don't know how feasible that is. It would require a new rendering of the page, at least.
- euank 12y ago
  That's not a solution for the general problem of getPixel + iframes though. Here, I'll give a simple example. Let's say there's a site that uses cookies to track logins. For example, hacker news. Now, hacker news does have the X-Frame-Option Deny, but let's assume it doesn't. So to figure out my hacker news username, all you have to do now is create an iframe with hacker news, and then getPixel on the area of the frame that contains usernames, run some trivial OCR, and done. Now, this issue is even more serious in other instances. For example, google talk widgets are embedded by iframes I believe. In both these cases, a fresh rendering would still have the inappropriate material due to cookies. If cookies aren't sent, e.g. you do a fresh render in a private tab, it would still have security concerns for anything that displays different, sometimes private, content based on ip address. For an example of that, you could render "private.internal.company.localsite" in an iframe and if a visitor from that company visited, even a cookie-less load would probably show private data due to the internal site relying on ip/nat controls.
  - comex 12y ago
    Any reason you couldn't just outright black out any pixels covered by different-origin iframes?
    - jgraham 12y ago
      Even some same-origin things, for example input type=file, which can reveal paths on the local filesystem, would have to be blanked out.
  - tlrobinson 12y ago
    There's no reason you couldn't apply the exact same "same origin policy" to elements (iframes, images, etc) when rendering into a bitmap.
    - euank 12y ago
      You could, but it would change scope a little. Right now, it's reasonable to expect that information will not be leaked to external servers, but user interactions can be faked. On an information only webpage (no user interactions) there's no need to have the X-Frame-Options to remain secure. If there's a way to access the data in that frame suddenly, that changes the necessity of that header.
      - tlrobinson 12y ago
        I'm not sure what X-Frame-Options has to do with it. I'm suggesting either not allowing rendering of bitmaps containing different origin iframes/images, or blanking out those elements in the rendering. Canvas does exactly that by not allowing different origin images to be drawn into a canvas element.
jstsch 12y ago
Hmm, if I can get all underlying DOM, including CSS, what is the difference except massive convenience? I could imagine iframes of other origins to be blank, but that would be it, I think?
- mbrubeck 12y ago
  Also cross-origin images, which would affect a large number of sites. And visited link coloring [1], although I supposed the browser could re-render the page without it for screenshot purposes. 1. http://dbaron.org/mozilla/visited-privacy http://dbaron.org/mozilla/visited-privacy

spankalee 12y ago

Firefox has had the element() CSS function for a while, but they're the only ones. https://developer.mozilla.org/en-US/docs/Web/CSS/element https://developer.mozilla.org/en-US/docs/Web/CSS/element I've written GUI builders that really could have used this for icons of views. Instead I had to render on the server. It'd be nice if it got more broadly adopted.