5 ms·

Generating Images in JavaScript Without Using the Canvas API

106 points by petercooper 9y ago

Xoros 9y ago
Despite the fact that it's pretty good stuff, shouldn't the title says it's for Android notification ?
- _alastair 9y ago
  Author here. The post is a combination of two separate things: creating a PNG file via JavaScript typed arrays, then using that PNG in an (Android-specific) web notification. I added a subtitle to the Medium post, but should have considered the fact that it won't show up anywhere else. Not sure it would fit if it was appended to the end of the title here.
- andrewla 9y ago
  That was the goal, but the underlying library is general-purpose. Writing a minimal uncompressed PNG writer from scratch is a fun exercise; the spec is pretty clear and the only tricky part, really, is getting the uncompressed zlib part right, especially since the block sizes are undocumentedly little-endian, while everything else is big-endian. Self-contained code for image generation and consumption can be pretty useful. It's one of the reasons that I like farbfeld, suckless's incredible simple binary image format, because libraries for processing it are essentially unnecessary since the format itself is so straightforward.
  - _alastair 9y ago
    > especially since the block sizes are undocumentedly little-endian, while everything else is big-endian. Yes! That stumped me for a long time. Along the way to solving it I found this great Stack Overflow answer where the creator of ADLER32 popped in to give some advice: https://stackoverflow.com/questions/32633486/why-are-png-deflate-block-lengths-being-ignored#comment53120790_32635822 https://stackoverflow.com/questions/32633486/why-are-png-def... "Also just realized that you are that Mark Adler."
- petercooper 9y ago
  That's a crucial point, though it doesn't affect how interesting the piece is. But since HN policy is to keep the original headline whatever it is I don't bother tweaking them anymore.
finchisko 9y ago
Hat off. Nice piece of engineering thinking.
zbjornson 9y ago
Cool. Nice, creative work. Indexed PNGs seem unfortunately under utilized. I happen to have just opened a PR to support indexed PNGs in node-canvas (HTML Canvas impl for node) based roughly on the proposed pixelFormat API [1,2] for the same reason that the space savings can be immense. It's also straightforward to change the palette of an indexed PNG after it's encoded because you don't need to re-compress the body. This lets you do cool tricks with recoloring images. [1] https://github.com/Automattic/node-canvas/pull/935 https://github.com/Automattic/node-canvas/pull/935 [2] https://github.com/WICG/canvas-color-space/blob/master/CanvasColorSpaceProposal.md https://github.com/WICG/canvas-color-space/blob/master/Canva...
- TazeTSchnitzel 9y ago
  Decent image editors etc do often output indexed PNGs. And thankfully, modern viewers do actually support them.
tambourine_man 9y ago
I handcrafted GIFs back in the day, mostly by looking at generated files and fiddling with the hex. Lots of fun. I'm curious if it would've been simpler to use it, albeit less efficient.
deleted 9y ago
[deleted]
moron4hire 9y ago
I had done a similar thing for PCM WAV files before Web Audio API was really a thing: http://www.seanmcbeth.com/synth/ http://www.seanmcbeth.com/synth/ Pretty sure that only runs in Chrome anymore.
seanwilson 9y ago
Maybe I'm missing something but why not use SVG for some of this? Would be perfect for the lozenge pictures.
- _alastair 9y ago
  You can't use an SVG as an image in a notification. A real shame as it would have solved all the device pixel ratio problems, but the SVG spec is absolutely huge (and includes stuff like animation) so I'm not overly surprised that Android doesn't support it.
  - seanwilson 9y ago
    Hmm, I'm wondering if there's a way now to render an SVG (not the full spec) within a service worker e.g. using PngPong with https://github.com/canvg/canvg https://github.com/canvg/canvg
    - _alastair 9y ago
      Huh! That's a really fascinating library. My gut instinct is still that it would be a heavy lift, though. Without the Canvas API in the middle you'd need to reimplement pixel by pixel drawing of different shape/curves (PngPong literally only draws squares and copies sections of other images) anti-aliasing, etc. etc. It would be a fascinating thing to work on, but the OffscreenCanvas API is coming to Chrome[1], so I suspect it would end up being a lot of work that would be obselete after not all that much time. [1] https://www.chromestatus.com/feature/5424182347169792 https://www.chromestatus.com/feature/5424182347169792
      - joshmarinacci 9y ago
        This might work for you. pure JS impl of canvas https://github.com/joshmarinacci/node-pureimage https://github.com/joshmarinacci/node-pureimage
        _alastair 9y ago
VikingCoder 9y ago
I can't even tell you how many times I've re-implemented PPM file format in whatever language, to produce an image file, when no libraries were available to me. Dozens, easily.
- striking 9y ago
  That's an excellent idea. I wonder if it'd work with this scenario.
mnutt 9y ago
It's a really clever solution, kudos to them. Is there a limitation why they couldn't just have the notification point to a remote URL and update server-side using whatever solution they wanted? It wouldn't have been as elegant but may have been easier to build.
- _alastair 9y ago
  The main reason is that the phone will download the image each time you post a notification, and with 650 alerts throughout the night (for 650 constituencies) the user would end up downloading a lot of data with very little to show for it (since newer notifications replace the older ones). But to add to that, it needed to be customised for different screen sizes. So we'd have to have generated a lot of versions server side.
18nleung 9y ago
I tried making PNGs before from bytes but got stuck on CRC generation - the math just seemed way over my head. Is there a simple way to explain CRC for someone with just a high school mathematics background?
- kuschku 9y ago
  Depends on where you went to high school. Did you have polynom division in school (most people have)? In that case, Wikipedia should be enough. Otherwise, you’d probably have to learn some more advanced math first to understand CRC well enough to implement it.
- karlding 9y ago
  I've always found "A Painless Guide to CRC Error Detection Algorithms" [1] from the zlib site pretty useful and easy to understand. Once you understand polynomial division, things start making a lot more sense. [1] https://zlib.net/crc_v3.txt https://zlib.net/crc_v3.txt
  - 18nleung 9y ago
    Thank you, this looks great.
bradneuberg 9y ago
Nice creative hackery in the browser! Always great to see well written blog posts on pushing web browsers in new unexpected directions.

106 points by petercooper 9y ago

Xoros 9y ago

Despite the fact that it's pretty good stuff, shouldn't the title says it's for Android notification ?

_alastair 9y ago
Author here. The post is a combination of two separate things: creating a PNG file via JavaScript typed arrays, then using that PNG in an (Android-specific) web notification. I added a subtitle to the Medium post, but should have considered the fact that it won't show up anywhere else. Not sure it would fit if it was appended to the end of the title here.
andrewla 9y ago
That was the goal, but the underlying library is general-purpose. Writing a minimal uncompressed PNG writer from scratch is a fun exercise; the spec is pretty clear and the only tricky part, really, is getting the uncompressed zlib part right, especially since the block sizes are undocumentedly little-endian, while everything else is big-endian. Self-contained code for image generation and consumption can be pretty useful. It's one of the reasons that I like farbfeld, suckless's incredible simple binary image format, because libraries for processing it are essentially unnecessary since the format itself is so straightforward.
- _alastair 9y ago
  > especially since the block sizes are undocumentedly little-endian, while everything else is big-endian. Yes! That stumped me for a long time. Along the way to solving it I found this great Stack Overflow answer where the creator of ADLER32 popped in to give some advice: https://stackoverflow.com/questions/32633486/why-are-png-deflate-block-lengths-being-ignored#comment53120790_32635822 https://stackoverflow.com/questions/32633486/why-are-png-def... "Also just realized that you are that Mark Adler."
petercooper 9y ago
That's a crucial point, though it doesn't affect how interesting the piece is. But since HN policy is to keep the original headline whatever it is I don't bother tweaking them anymore.

finchisko 9y ago

Hat off. Nice piece of engineering thinking.

zbjornson 9y ago

Cool. Nice, creative work. Indexed PNGs seem unfortunately under utilized. I happen to have just opened a PR to support indexed PNGs in node-canvas (HTML Canvas impl for node) based roughly on the proposed pixelFormat API [1,2] for the same reason that the space savings can be immense. It's also straightforward to change the palette of an indexed PNG after it's encoded because you don't need to re-compress the body. This lets you do cool tricks with recoloring images. [1] https://github.com/Automattic/node-canvas/pull/935 https://github.com/Automattic/node-canvas/pull/935 [2] https://github.com/WICG/canvas-color-space/blob/master/CanvasColorSpaceProposal.md https://github.com/WICG/canvas-color-space/blob/master/Canva...

TazeTSchnitzel 9y ago
Decent image editors etc do often output indexed PNGs. And thankfully, modern viewers do actually support them.

tambourine_man 9y ago

I handcrafted GIFs back in the day, mostly by looking at generated files and fiddling with the hex. Lots of fun. I'm curious if it would've been simpler to use it, albeit less efficient.

deleted 9y ago

[deleted]

moron4hire 9y ago

I had done a similar thing for PCM WAV files before Web Audio API was really a thing: http://www.seanmcbeth.com/synth/ http://www.seanmcbeth.com/synth/ Pretty sure that only runs in Chrome anymore.

seanwilson 9y ago

Maybe I'm missing something but why not use SVG for some of this? Would be perfect for the lozenge pictures.

_alastair 9y ago
You can't use an SVG as an image in a notification. A real shame as it would have solved all the device pixel ratio problems, but the SVG spec is absolutely huge (and includes stuff like animation) so I'm not overly surprised that Android doesn't support it.
- seanwilson 9y ago
  Hmm, I'm wondering if there's a way now to render an SVG (not the full spec) within a service worker e.g. using PngPong with https://github.com/canvg/canvg https://github.com/canvg/canvg
  - _alastair 9y ago
    Huh! That's a really fascinating library. My gut instinct is still that it would be a heavy lift, though. Without the Canvas API in the middle you'd need to reimplement pixel by pixel drawing of different shape/curves (PngPong literally only draws squares and copies sections of other images) anti-aliasing, etc. etc. It would be a fascinating thing to work on, but the OffscreenCanvas API is coming to Chrome[1], so I suspect it would end up being a lot of work that would be obselete after not all that much time. [1] https://www.chromestatus.com/feature/5424182347169792 https://www.chromestatus.com/feature/5424182347169792
    - joshmarinacci 9y ago
      This might work for you. pure JS impl of canvas https://github.com/joshmarinacci/node-pureimage https://github.com/joshmarinacci/node-pureimage
      - _alastair 9y ago

VikingCoder 9y ago

I can't even tell you how many times I've re-implemented PPM file format in whatever language, to produce an image file, when no libraries were available to me. Dozens, easily.

striking 9y ago
That's an excellent idea. I wonder if it'd work with this scenario.

mnutt 9y ago

It's a really clever solution, kudos to them. Is there a limitation why they couldn't just have the notification point to a remote URL and update server-side using whatever solution they wanted? It wouldn't have been as elegant but may have been easier to build.

_alastair 9y ago
The main reason is that the phone will download the image each time you post a notification, and with 650 alerts throughout the night (for 650 constituencies) the user would end up downloading a lot of data with very little to show for it (since newer notifications replace the older ones). But to add to that, it needed to be customised for different screen sizes. So we'd have to have generated a lot of versions server side.

18nleung 9y ago

I tried making PNGs before from bytes but got stuck on CRC generation - the math just seemed way over my head. Is there a simple way to explain CRC for someone with just a high school mathematics background?

kuschku 9y ago
Depends on where you went to high school. Did you have polynom division in school (most people have)? In that case, Wikipedia should be enough. Otherwise, you’d probably have to learn some more advanced math first to understand CRC well enough to implement it.
karlding 9y ago
I've always found "A Painless Guide to CRC Error Detection Algorithms" [1] from the zlib site pretty useful and easy to understand. Once you understand polynomial division, things start making a lot more sense. [1] https://zlib.net/crc_v3.txt https://zlib.net/crc_v3.txt
- 18nleung 9y ago
  Thank you, this looks great.

bradneuberg 9y ago

Nice creative hackery in the browser! Always great to see well written blog posts on pushing web browsers in new unexpected directions.