4 ms·

> What if we could also vectorize 2D <canvas> elements controlled by JavaScript? Turns out, Chromium has this capability built-in for printing: I'm very surpri

> What if we could also vectorize 2D <canvas> elements controlled by JavaScript? Turns out, Chromium has this capability built-in for printing: I'm very surprised to hear this. So printing, either to PDF or to actual printers, may reveal more information about what was drawn to the canvas than normal display, especially if no effort has been made to remove overdrawn paint records. That can have an interesting, if only hypothetical, consequence...

andybak 4y ago
Being fair, if anyone is sending anything to the client and assumes it's not visible then they are fair game. I just hope the code around password entry fields is carefully audited. That's all on the client.
- version_five 4y ago
  There's a story from years ago (I couldn't find it) about some government or legal documents having info redacted, but whoever did it just used some pdf editing tool to draw black boxes over the redacted parts, so all the info was still in the pdfs. Edit: I found this but I'm not sure it's the one I'm remembering: https://www.techdirt.com/2014/01/28/new-york-times-suffers-redaction-failure-exposes-name-nsa-agent-targeted-network-uploaded-pdf/ https://www.techdirt.com/2014/01/28/new-york-times-suffers-r...
  - perth 4y ago
    This happened with the Maxwell redacted court docs
    - lliamander 4y ago
      Do you have a link to the unrelated docs?
      - lliamander 4y ago
        *unredacted Stupid autocorrect
  - karlshea 4y ago
    One of the Apple v Samsung (?) cases had court docs released with some App Store numbers redacted but it was just black squares over the actual numbers, just selecting the sentence and copy/pasting it gave it to you.
- kevincox 4y ago
  Yes, but there is more than that there. What if I as the client try to print a page or export to PDF. I think that there is nothing sensitive visible on the page so I share the result. It turns out that there was actually sensitive info in the canvas that was not visible due to something like overdraw. As a simple example imagine that an image is drawn to the canvas and then blacked out. You wouldn't expect that the saved PDF may contain those as separate layers. Of course this highlights an existing issue with complex formats. You need to be very careful before sharing complex documents.
- rkagerer 4y ago
  I've seen big-firm lawyers make the same mistake more often than you'd think - a pdf "redaction" that's just a black box under which you can simply highlight and copy the obscured text. Oops.
tyingq 4y ago
Pretty sure canvas.toDataURL() is or was a fingerprinting method.
- piskerpan 4y ago
  Kinda unrelated. That API exports a flattened PNG
mk_stjames 4y ago
This is exactly the case. I've done conversions before where it was possible to see and extract underlying, hidden elements, that were not visible or even detectable in the rendered webpage in a browser. This is actually a somewhat common method when it comes to a bit of corporate sleuthing.. anytime you see a pretty website with vector-y graphics, maybe engineering-drawing representations.. if the data hasn't been stripped completely or redrawn you can extract information that otherwise people would assume unknowable. In a recent example... I did this on a startup company's page involving a product where they had a CAD-like side view drawing of one of their products... but the base file (in this case it was an SVG) driving the page actually contained multiple hidden views of the same product and other products and at the 'real' precision of what likely was a DXF export from a CAD program, given to the web team. This allowed a critical dimension of an unannounced product to be precisely determined (to three significant figures) which was a spec that had not been publicly released...
- LoganDark 4y ago
  > In a recent example... I did this on a startup company's page involving a product where they had a CAD-like side view drawing of one of their products... but the base file (in this case it was an SVG) driving the page actually contained multiple hidden views of the same product and other products and at the 'real' precision of what likely was a DXF export from a CAD program, given to the web team. This allowed a critical dimension of an unannounced product to be precisely determined (to three significant figures) which was a spec that had not been publicly released... This is actually really interesting. Do you get to do this often?
aidos 4y ago
Totally. We see architectural drawings that go through a number of revisions and it’s not uncommon for designers to simply cover a whole section with a white box and then draw on top of it. Also within PDFs (and svgs) you normally clip the area you’re going to draw into to bound it (sort of like overflow:hidden) and anything outside of that doesn’t display, but it’s still there and accessible. I marvel more at the fact that software is capable of figuring out all the occlusions so you can print the stuff on a plotter. Cad drawings have up to 2M individual vectors in them. Its impressive that it works at all to be honest.
deleted 4y ago
[deleted]
eyelidlessness 4y ago
> > What if we could also vectorize 2D <canvas> elements controlled by JavaScript? […] > I'm very surprised to hear this. So printing, either to PDF or to actual printers, may reveal more information about what was drawn to the canvas than normal display, especially if no effort has been made to remove overdrawn paint records. That can have an interesting, if only hypothetical, consequence... The canvas API is all imperative code, so you might think it’s fairly opaque. That’s what I thought anyway, until recently I hacked on a someone’s generated art demo, mostly to glean insights into the algorithms used. As I looked through the actual canvas-specific code, it struck me that 1) it’s exceedingly statically analyzable and 2) that the imperative APIs could trivially translate to an incremental SVG rendering, because their primitives are nearly identical apart from the imperative/declarative distinction. Mentioning this mainly because if there’s anything interesting to learn about a particular usage of canvas, it would probably not be a huge investment to learn it. Either by static analysis or by rendering canvas calls incrementally to SVG, anything overdrawn or obscured is sitting right there to inspect without any special browser-internal faculties.
vbezhenar 4y ago
I don't really understand this paragraph. I just created test.html: <!DOCTYPE html> <canvas id="tutorial" width="150" height="150"></canvas> <script> var c = document.getElementById("tutorial"); var ctx = c.getContext("2d"); ctx.beginPath(); ctx.moveTo(75, 50); ctx.lineTo(100, 75); ctx.lineTo(100, 25); ctx.fill(); </script> and used Chrome to pdf print it. Then I opened PDF in a Sumatra PDF Reader, zoomed in and it's obviously not vectorized.
- lifthrasiir 4y ago
  That's a good point, and it took me a bit of trials to force Chrome to print canvas in vector. The article links to the MDN <canvas> demo [1], which doesn't print in vector if you try it. But you can run the following in the console and then try to print the same page, which will be vectorized: window.onbeforeprint = () => { window.canvas.width |= 0; // clear the canvas draw(); }; So I guess, for now, this behavior only manifests when you use the beforeprint event. [1] https://yari-demos.prod.mdn.mozit.cloud/en-US/docs/Web/API/Canvas_API/Tutorial/Drawing_shapes/_sample_.making_combinations.html https://yari-demos.prod.mdn.mozit.cloud/en-US/docs/Web/API/C...
  - vbezhenar 4y ago
    Thanks a lot! Just a little typo: it should be `onbeforeprint` I guess. I just checked and it works. That approach has a potential to solve important problem for me.
    - lifthrasiir 4y ago
      Oh, I had a trouble in spelling "print" vs. "paint" ;-) (Fixed.) I now wonder why Blink specifically supports this, presumably to support some weird websites?
- afiori 4y ago
  I read it as "convert to SVG" rather than "reconstruct vector data"

andybak 4y ago

Being fair, if anyone is sending anything to the client and assumes it's not visible then they are fair game. I just hope the code around password entry fields is carefully audited. That's all on the client.

version_five 4y ago
There's a story from years ago (I couldn't find it) about some government or legal documents having info redacted, but whoever did it just used some pdf editing tool to draw black boxes over the redacted parts, so all the info was still in the pdfs. Edit: I found this but I'm not sure it's the one I'm remembering: https://www.techdirt.com/2014/01/28/new-york-times-suffers-redaction-failure-exposes-name-nsa-agent-targeted-network-uploaded-pdf/ https://www.techdirt.com/2014/01/28/new-york-times-suffers-r...
- perth 4y ago
  This happened with the Maxwell redacted court docs
  - lliamander 4y ago
    Do you have a link to the unrelated docs?
    - lliamander 4y ago
      *unredacted Stupid autocorrect
- karlshea 4y ago
  One of the Apple v Samsung (?) cases had court docs released with some App Store numbers redacted but it was just black squares over the actual numbers, just selecting the sentence and copy/pasting it gave it to you.
kevincox 4y ago
Yes, but there is more than that there. What if I as the client try to print a page or export to PDF. I think that there is nothing sensitive visible on the page so I share the result. It turns out that there was actually sensitive info in the canvas that was not visible due to something like overdraw. As a simple example imagine that an image is drawn to the canvas and then blacked out. You wouldn't expect that the saved PDF may contain those as separate layers. Of course this highlights an existing issue with complex formats. You need to be very careful before sharing complex documents.
rkagerer 4y ago
I've seen big-firm lawyers make the same mistake more often than you'd think - a pdf "redaction" that's just a black box under which you can simply highlight and copy the obscured text. Oops.

tyingq 4y ago

Pretty sure canvas.toDataURL() is or was a fingerprinting method.

piskerpan 4y ago
Kinda unrelated. That API exports a flattened PNG

mk_stjames 4y ago

This is exactly the case. I've done conversions before where it was possible to see and extract underlying, hidden elements, that were not visible or even detectable in the rendered webpage in a browser. This is actually a somewhat common method when it comes to a bit of corporate sleuthing.. anytime you see a pretty website with vector-y graphics, maybe engineering-drawing representations.. if the data hasn't been stripped completely or redrawn you can extract information that otherwise people would assume unknowable. In a recent example... I did this on a startup company's page involving a product where they had a CAD-like side view drawing of one of their products... but the base file (in this case it was an SVG) driving the page actually contained multiple hidden views of the same product and other products and at the 'real' precision of what likely was a DXF export from a CAD program, given to the web team. This allowed a critical dimension of an unannounced product to be precisely determined (to three significant figures) which was a spec that had not been publicly released...

LoganDark 4y ago
> In a recent example... I did this on a startup company's page involving a product where they had a CAD-like side view drawing of one of their products... but the base file (in this case it was an SVG) driving the page actually contained multiple hidden views of the same product and other products and at the 'real' precision of what likely was a DXF export from a CAD program, given to the web team. This allowed a critical dimension of an unannounced product to be precisely determined (to three significant figures) which was a spec that had not been publicly released... This is actually really interesting. Do you get to do this often?

aidos 4y ago

Totally. We see architectural drawings that go through a number of revisions and it’s not uncommon for designers to simply cover a whole section with a white box and then draw on top of it. Also within PDFs (and svgs) you normally clip the area you’re going to draw into to bound it (sort of like overflow:hidden) and anything outside of that doesn’t display, but it’s still there and accessible. I marvel more at the fact that software is capable of figuring out all the occlusions so you can print the stuff on a plotter. Cad drawings have up to 2M individual vectors in them. Its impressive that it works at all to be honest.

deleted 4y ago

[deleted]

eyelidlessness 4y ago

> > What if we could also vectorize 2D <canvas> elements controlled by JavaScript? […] > I'm very surprised to hear this. So printing, either to PDF or to actual printers, may reveal more information about what was drawn to the canvas than normal display, especially if no effort has been made to remove overdrawn paint records. That can have an interesting, if only hypothetical, consequence... The canvas API is all imperative code, so you might think it’s fairly opaque. That’s what I thought anyway, until recently I hacked on a someone’s generated art demo, mostly to glean insights into the algorithms used. As I looked through the actual canvas-specific code, it struck me that 1) it’s exceedingly statically analyzable and 2) that the imperative APIs could trivially translate to an incremental SVG rendering, because their primitives are nearly identical apart from the imperative/declarative distinction. Mentioning this mainly because if there’s anything interesting to learn about a particular usage of canvas, it would probably not be a huge investment to learn it. Either by static analysis or by rendering canvas calls incrementally to SVG, anything overdrawn or obscured is sitting right there to inspect without any special browser-internal faculties.

vbezhenar 4y ago

I don't really understand this paragraph. I just created test.html: <!DOCTYPE html> <canvas id="tutorial" width="150" height="150"></canvas> <script> var c = document.getElementById("tutorial"); var ctx = c.getContext("2d"); ctx.beginPath(); ctx.moveTo(75, 50); ctx.lineTo(100, 75); ctx.lineTo(100, 25); ctx.fill(); </script> and used Chrome to pdf print it. Then I opened PDF in a Sumatra PDF Reader, zoomed in and it's obviously not vectorized.

lifthrasiir 4y ago
That's a good point, and it took me a bit of trials to force Chrome to print canvas in vector. The article links to the MDN <canvas> demo [1], which doesn't print in vector if you try it. But you can run the following in the console and then try to print the same page, which will be vectorized: window.onbeforeprint = () => { window.canvas.width |= 0; // clear the canvas draw(); }; So I guess, for now, this behavior only manifests when you use the beforeprint event. [1] https://yari-demos.prod.mdn.mozit.cloud/en-US/docs/Web/API/Canvas_API/Tutorial/Drawing_shapes/_sample_.making_combinations.html https://yari-demos.prod.mdn.mozit.cloud/en-US/docs/Web/API/C...
- vbezhenar 4y ago
  Thanks a lot! Just a little typo: it should be `onbeforeprint` I guess. I just checked and it works. That approach has a potential to solve important problem for me.
  - lifthrasiir 4y ago
    Oh, I had a trouble in spelling "print" vs. "paint" ;-) (Fixed.) I now wonder why Blink specifically supports this, presumably to support some weird websites?
afiori 4y ago
I read it as "convert to SVG" rather than "reconstruct vector data"