7 ms·
CanvasKit – Skia and WebAssembly
- espeed 7y agoHere's one of the Skia Path Ops videos referenced in the Presentation section -- for some reason it links to slides but not the videos... Skia Path Ops : High Performance Set Operations for Geometry https://www.youtube.com/watch?v=OmfliNQsk88 https://www.youtube.com/watch?v=OmfliNQsk88
- skybrian 7y agoIt sounds like this allows you to draw to a WebGL surface as if it were Canvas?
- espeed 7y agoIt means you can now include Chrome's Skia vector graphics rendering engine as part of your custom code in WebAssembly (skia/wasm in Chrome or elsewhere). See the Slug thread from earlier today for context... Slug: Dynamic GPU Font Rendering and Advanced Text Layout https://news.ycombinator.com/item?id=20475111 https://news.ycombinator.com/item?id=20475111 https://en.wikipedia.org/wiki/Skia_Graphics_Engine https://en.wikipedia.org/wiki/Skia_Graphics_Engine
- sansnomme 7y agoWould be terrible if a browser decides to "optimize" the Wasm and upon detection of code that it has natively, directly runs the native version and completely bypasses any security benefits. (If you work on Skia and are reading this, please DON'T compromise on security. We already have too many vulnerabilities on the hardware level thanks to relentless optimizations that was not or unable to be proved correct.)
- kevingadd 7y agoNo browser would ever do this. It's a pain in the ass to do to begin with but it's also an absurdly complex optimization for a specific use case that would barely pay off in practice. It wouldn't work unless you were shipping the same version of skia as the embedded application and you'd have to do all sorts of weird stuff to make the ABIs match up (the native C/C++ ABI is not the same as the one wasm uses at runtime)
- bogwog 7y agoKnowing Google, they might be willing to go through the effort so that Skia sites only work in Chrome.
- skybrian 7y agoRight, but I'm wondering if it runs entirely on the CPU or uses GPU-based acceleration via WebGL.
- espeed 7y agoThe docs say it uses WebGL [1]. And I found a "fixed" pull-request saying it too [2]. WebGL context encapsulated as an SkSurface, allowing for direct drawing to an HTML canvas. However, I just discovered this project a few hours ago so I don't know if there's any "gotchas" -- hopefully someone more knowledgeable or on the chrome/skia team will chime in. [1] https://skia.org/user/modules/canvaskit https://skia.org/user/modules/canvaskit [2] https://bugs.chromium.org/p/skia/issues/detail?id=8378 https://bugs.chromium.org/p/skia/issues/detail?id=8378
- Lerc 7y agoIt is certainly a WebGL Context. Running this on one of the examples. let c=document.querySelector("#canvas"); c.getContext("2d"); >>>> null c.getContext("webgl"); >>>> WebGLRenderingContext { vertexAttribDivisor: vertexAttribDivisor(), drawArraysInstanced: drawArraysInstanced(), drawElementsInstanced: drawElementsInstanced(), createVertexArray: createVertexArray(), deleteVertexArray: deleteVertexArray(), bindVertexArray: bindVertexArray(), isVertexArray: isVertexArray(), drawBuffers: drawBuffers(), Yu: null, canvas: canvas#canvas } c.getContext("webgl2"); >>>> null Of course that doesn't stop it from using any amount of software rendering layers before it hits the canvas, but it's a good sign.
- rikroots 7y agoAs far as I'm aware, browsers have been making use of any available GPU acceleration for rendering 2d contexts since 2012[1]? The example supplied by Google[2] (on the first link) definitely requests a 2d context for the canvas. [1] - https://developers.google.com/web/updates/2012/07/Taking-advantage-of-GPU-acceleration-in-the-2D-canvas https://developers.google.com/web/updates/2012/07/Taking-adv... [2] - http://fhtr.org/gravityring/sprites.html http://fhtr.org/gravityring/sprites.html
- dvh 7y agoIt takes 1 minute to draw star (first demo). Why is it so slow?
- aidos 7y agoNot for me and I’m on mobile.
- zamadatix 7y agoThere are a few MB of .wasm included in the page. Takes about half a second to load it and another half second to initialize that for me on Chrome/FF. If it's not connection speed related then there is probably something buggy going on with running WASM in your browser.
- srameshc 7y agoYesterday I saw a post about tinygo (tinygo.org) which can produce WebAssembly code and now Skia + WebAssembly. Fun times are ahead with Go :)
- pier25 7y agoIt's less than 50KB Gzipped, not bad at all. https://bundlephobia.com/result?p=canvaskit-wasm@0.6.0 https://bundlephobia.com/result?p=canvaskit-wasm@0.6.0
- kllrnohj 7y agoI don't think that's accurate at all. Watching Chrome's network profiler it's pulling down 2.5MB .wasm for the star fiddle: https://jsfiddle.skia.org/canvaskit/ea89749ae8c90bce807ea2e7e34fb7b09b950cee70d9db0a9cdfd2d67bd48ef0 https://jsfiddle.skia.org/canvaskit/ea89749ae8c90bce807ea2e7... 50KB Gzipped seems like it's just the wrapped JS code, not the .wasm code.
- pier25 7y agoThe NPM package doesn't list any dependencies: https://github.com/google/skia/blob/master/modules/canvaskit/package.json https://github.com/google/skia/blob/master/modules/canvaskit... I'm not saying you are wrong, but if the NPM package needed more stuff it would be in the package.json, no?
- kllrnohj 7y agoI'm more saying bundlephobia.com is wrong in that it doesn't appear to be including .wasm files at all. If you just do an npm install of canvaskit-wasm you'll find a 6.4M canvaskit.wasm file. That'd have to be some insane gzip magic to bring that down to 50Kb. If I gzip the entire canvaskit module (including the font files) I'm getting a size of 3.4M. If I just gzip the .wasm file I get a size of 2.4M.
- pier25 7y agoThanks for checking that out. You are right of course.
- pier25 7y agoAnyone knows why text looks so blurry? Where are the docs for CanvasKit?
- kllrnohj 7y ago> Anyone knows why text looks so blurry? If you're on desktop it's probably because it's not doing any form of subpixel hinting or AA. It's both harder to do that in a GL context, and I don't know if there's even any way to get the display's subpixel layout in JS in the first place.
- pier25 7y agoIt looks like there is antialiasing, although it's huge. It only happens for text resizing demo though, the text on the star demo looks better.
- kllrnohj 7y agoIt should be doing AA but not subpixel AA (aka, it's not going to match cleartype). Those are 2 different things.
- deleted 7y ago[deleted]
- deleted 7y ago[deleted]
- cgijoe 7y agoI'm guessing @pier25 is viewing on a retina (2X pixel density) screen. All the demos are rendered using 1X canvases, so they appear blurry on retina (the default is bicubic interp).
- pier25 7y agoYep, I'm on retina. But the vector shapes and the text on the star demo look better than the text shape demo.
- alleycat5000 7y agoI wonder if this is in part driven by Flutter for Web?
- espeed 7y agoSee the Skia/Flutter note below. Skia has been open-source since 2008 -- it's the wasm bit that's new. https://skia.org/dev/flutter https://skia.org/dev/flutter
- alleycat5000 7y agoRight, I just mean specifically the new web stuff in Flutter: https://flutter.dev/web https://flutter.dev/web Presumably having Skia available in the browser makes supporting the web platform easier.
- hajile 7y agoI can't think of another reason for Google to pour in all those resources (Google apparently being the biggest sponsor and owner of Skia). This could also more generally help give native ChromeOS apps a much needed kick (instead of needing Android apps for so many things).
- markdog12 7y agoThey should use the desynchronized canvas flag for lower latency on the ink example. canvas.getContext('webgl', {desynchronized: true}) https://www.chromestatus.com/feature/6360971442388992 https://www.chromestatus.com/feature/6360971442388992
- mkl 7y agoThat seems like it would have to happen within CanvasKit itself, as the code using it only uses abstractions over the canvas itself. I can't find anywhere in CanvasKit where it does getContext('webgl'), only '2d', so I'm not sure how it's doing it. The latency of that example seems great with a mouse, and a little laggy with a pen (a bit better than a pure canvas drawing app I've been working on). The frame rate is much better than my app.
- atomicUpdate 7y agoThis page is infuriating to use on my phone. I can barely scroll, and when I do get lucky, it bounces back to the top and more things load.
- lowtolerance 7y agoWeb development-centric websites often seem to totally ignore the need to provide a usable mobile experience. It’s maddeningly weird.
- acoard 7y agoIt's not unsurprising. Web development happens practically exclusively on desktop/laptops.
- zapzupnz 7y agoAnd yet, desktop browsers have no more need of hijacking the browser's scrolling behaviour than any other browser.
- deleted 7y ago[deleted]
- paulryanrogers 7y agoAnd leave scroll bar rendering to the browser or OS vendors!? Think of the pixels! /s Seriously though I believe it's often done to make infinite scrolling and scroll-transitions more visually pleasant. (Not that I agree with the sentiment, but if visuals are more important than anything else...)
- Waterluvian 7y agoMaybe I'm a small ignorable minority but almost all my library skimming is done on a phone. I explore new technologies when rocking babies to bed or dozing off myself.
- pault 7y ago
- c-smile 7y agoCool, but why? Any browser already includes the very same set of graphic primitives that Skia provides. Just to render what you see on the screen right now. Why not just to expose all that as API for WASM or whatever?
- kevingadd 7y agoHTML canvas is incredibly slow and just a generally bad API. Using it from WASM is a mistake. It uses strings for everything and has a particularly slow imperative API, and the spec/de-facto-spec behaviors perform very slowly even on high-end hardware. I've seen relatively simple 2D canvas-based games drop frames in chrome on a GTX 2080ti because canvas is that bad.
- c-smile 7y ago"HTML canvas ... generally bad API" Agree on that. But this is not about Canvas::Graphics.
- kevingadd 7y agoThe original post appeared to be suggesting just using HTML Canvas (since it's backed by Skia). Was it suggesting something else?
- z3t4 7y agoThe slow downs usually comes from abstraction layers. If you just use the api it will be fast. But if you create 1 million new objects and have 2 million function calls in excess on every frame it will be slow. Eg. If you dont like imperative style it will be slow. A compilation step and a bunch of optimization runs could make anything fast thought.
- kevingadd 7y agoIt really is not. If you want to set the current color you have to assemble a css color string from your color values. There is no way to make this fast. When you ship production 2D games using canvas, stuff like this shows up in your profiles and there's no way to optimize it out other than not using canvas. (Many people switch to using WebGL to do 2D themselves for this reason - it's faster.)
- xfalcox 7y agoI just created a proposal to support lottie based animated stickers in Discourse two days ago, and this looks very useful to get more performance when playing those using the new Skottie module. Very cool! https://meta.discourse.org/t/plugin-for-animated-stickers/123212?u=falco https://meta.discourse.org/t/plugin-for-animated-stickers/12...
- bcheung 7y agoWhat is this for and why would someone want to use it? There's basically nothing on the website but the API. No decent overview or tutorial or even why someone would want to use this project.
- Secretmapper 7y agoI had a really hard time understanding it as well, and as someone who is supposed to at least have some understanding in this space (HTML5 gamedev) it made me feel a bit dumb. However, from what I understand, there seems to be two things needed to be noted here: Skia - a standardised library that can 'render' to different backends - i.e. it can render to OpenGL or SVG or PDFs. CanvasKit (the actual one linked) - basically use Skia on the web (with webgl). A huge boon here would be performance. For example, in WebGL, if you want to draw text, the current method is actually to render it in the canvas first, and reupload the texture to the GPU. From what I understand Skia seems to handle the rendering 'natively' so this step is not necessary. So basically it would just make it easier to draw 'primitives' since they are provided by the Skia API without having to do workarounds or writing really low level webgl fragments/shaders. This is just from what I gathered, but yeah it is confusing. If anyone knowledgeable reads this, let me know if I got anything wrong!
- bogwog 7y agoSkia is just a 2D drawing API, like Cairo. It's what Chrome uses under the hood to render their whole UI I think, including (non-WebGL) HTML5 canvas stuff. Why you would want to use this over HTML5 canvas? Good question. Maybe you want some features in Skia that aren't in the canvas API? Maybe you have a Skia app that you want to port to the web?
- pault 7y agoI can think of a use case. Writing a GPU accelerated text editor with syntax highlighting that can interface with a back end like neovim. Maybe not super useful but I could see myself doing this since my only expertise is with web technologies.
- leonidasv 7y agoSkia is a graphics engine developed in C++. Think of it as the building block for graphical interfaces, animations and everything else that involves displaying some 2D data on a screen, specially vectorial data. For example, Flutter uses it to draw its UIs and Chrome uses it for almost everything, including rendering text parsed from HTML. Sublime Text, Firefox, Xamarin and many other projects also rely on Skia for the same sort of thing. However, unlike, let's say, Qt or GTK+, Skia does not provide already done widgets (i.e., drop-in buttons, windows, text inputs etc.). It's like the fundamental building block on top of which you can create your buttons, animations and everything else and then display it on screen. Now, CanvasKit is Skia ported to the browser via WebAssembly. Since Skia is a mature project and very performant (as you can guess from who is using it), it's now an alternative to Canvas API and DOM to render things on websites and will also let existing native applications to be ported to the web more easily.
- aloer 7y agoWould be great if anyone here has some experience with this and examples of use cases. I have been looking at this for a few weeks now for a tile based view with multi threaded rendering and server side prerendering. But I just can't figure out what the benefits would be over canvas natively, if there even are any. As I wrote in https://news.ycombinator.com/item?id=20339574 https://news.ycombinator.com/item?id=20339574 " context: I want to build a tile based view (tiles in x/y dimensions + zoom levels). The content of the tiles is loaded from server (shapes mostly) and rendered into tile images client side. I also want the same tiles prerendered as identical images on the server. For this I have a feeling that something like skia is the way to go. Skia can be used via wasm bindings. How I would fetch the shapes and render the tiles (or fetch the prerendered ones) transparently and then where to render the tiles into, that is what I am trying to figure out. It feels like multithreading could be very useful here. Right now only chrome appears to support OffscreenCanvas (which can be accessed from webworkers), hence the idea of using skia directly and possibly going a level higher to write whatever kind of multithreaded render logic in rust and run it with wasm and a single "canvas output". Whether skia is the right choice here or not is also something I have yet to figure out The ultimate goal is quick startup (prerendered tiles) while simultaneously high performance when updating the entire view (=multiple tiles in parallel). This is mostly a learning project for me context: I want to build a tile based view (tiles in x/y dimensions + zoom levels). The content of the tiles is loaded from server (shapes mostly) and rendered into tile images client side. I also want the same tiles prerendered as identical images on the server. For this I have a feeling that something like skia is the way to go. Skia can be used via wasm bindings. How I would fetch the shapes and render the tiles (or fetch the prerendered ones) transparently and then where to render the tiles into, that is what I am trying to figure out. It feels like multithreading could be very useful here. Right now only chrome appears to support OffscreenCanvas (which can be accessed from webworkers), hence the idea of using skia directly and possibly going a level higher to write whatever kind of multithreaded render logic in rust and run it with wasm and a single "canvas output". Whether skia is the right choice here or not is also something I have yet to figure out The ultimate goal is quick startup (prerendered tiles) while simultaneously high performance when updating the entire view (=multiple tiles in parallel). This is mostly a learning project for me " Essentially what I am wondering is if using skia via wasm would allow me to bypass limits on canvas natively (and canvas in webworkers) so that I can render multiple tile images in parallel and then draw them into a single view for the user to interact with. Think google maps or similar but tiles rendered client side Overall this feels like there might be a better way and I am just not seeing it
- z3t4 7y agoI would like a Skia + v8 runtime. Some of my web apps are just a 2d canvas and JavaScript. Would be nice to be able to use them without a browser and bundle with something more lightweight then Electron.
- snek 7y agoI would assume the skia wasm blob can be integrated with cairo canvas, as it has the same exposed api.
- tapirl 7y agoCan anyone show the url of the source code (C API?) which is outputted to the wasm file? I mean the subfolder in the skia project.
- patientplatypus 7y agoCan someone explain to me what this drink example is doing? https://jsfiddle.skia.org/canvaskit/e7ac983d9859f89aff1b6d385190919202c2eb53d028a79992892cacceffd209 https://jsfiddle.skia.org/canvaskit/e7ac983d9859f89aff1b6d38... Just from a brief parse of the code I don't see how the drink could possibly be transforming the way that it is doing. The renderer - drawFrame in this case - isn't doing any heavy lifting to really make all the side animations (drink splash etc) occur. It appears that at least with this example, possibly the others (the lego one?) if they work they've hidden a lot of data in the fetched json data. Which seems not great. I mean the drink json (https://storage.googleapis.com/skia-cdn/misc/drinks.json https://storage.googleapis.com/skia-cdn/misc/drinks.json) is not human parse-able so I don't see how they made these examples at all. Maybe they made them in Adobe After Effects and then exported the keyframes as json or something, but there's no tutorial or directions that I can see. This is incredibly frustrating because a decent canvas tool is sorely needed (generally). Does anyone see what is going on? EDIT: So they do use lottie files (https://lottiefiles.com/410-lego-loader https://lottiefiles.com/410-lego-loader). But lottie files can already be exported to javascript incredibly easily, so unless you really need the extra performance of WASM this doesn't appear to add anything. Am I still missing something?