I have a modest proposal that might also work. Four canvases: 1. BW (inited from Color) 2. Color (color source image) 3. Paint (inited transparent) 4. Temp (i

I have a modest proposal that might also work. Four canvases: 1. BW (inited from Color) 2. Color (color source image) 3. Paint (inited transparent) 4. Temp (inited transparent) Each mouse move: 1. Draw an opaque white line on Paint to the new position 2. Copy Paint to Temp 3. Set the globalCompositeOperation to "source-in" 4. Draw Color to Temp 5. Set the globalCompositeOperation to "source-over" 6. Draw Temp to BW I haven't tried it, but I think it should work (or something like it with the flaws corrected :) ). One advantage is that you can use the path line drawing primitives on the canvas instead of rolling your own. You could also optimize a bit by only drawing the portions of the canvases that have been changed, but it really should be plenty fast as-is.