6 ms·

Servo WebRender Overview

191 points by dumindunuwan 11y ago

pcwalton 11y ago
The title on HN is very editorialized and I can't overemphasize how preliminary and experimental this work is. Take all numbers with a huge grain of salt. That said, I'm obviously really excited about this project. :) I feel strongly that it's the right from-scratch, modern approach for painting Web content. Happy to answer any questions.
- eridius 11y ago
  Wow this looks very cool. Kudos!
  - pcwalton 11y ago
    Glenn Watson deserves nearly all the credit here. My role has been limited to a little bit of high-level design input and messing around with signed distance fields and goofy stencil buffer tricks. :)
    - trishume 11y ago
      Ooooh signed distance fields, if you ever get a good Rust implementation going make sure to publish it and add a readme that at least says what the code is so that it can be found. It could be great for other rust libraries, apps and games. None of them have subpixel rendering at the moment AFAIK and that's super easy with distance fields and real important for appearance.
      - TD-Linux 11y ago
        >None of them have subpixel rendering at the moment AFAIK Actually, most Rust projects end up using Freetype for glyph rendering, which does support subpixel rendering. Signed distance fields are great for animated text, but they also have a couple of disadvantages - they generally produce lower quality output than Freetype at small glyph sizes, and the signed distance field is more complex to generate at start, which slows down initial page rendering. Given that 99% of web pages are static, it's a harder sell. Maybe some sort of heuristic based switching would work.
        constexpr 11y ago
        I'm surprised to hear they produce lower quality output at small glyph sizes. Fragment shaders can do multiple texture samples and I've gotten very high quality results. You can even do ClearType-style subpixel rendering with a handful of samples. The case where signed distance fields don't work is for large glyph sizes. They often end up chopping off corners or not having the density to represent thin calligraphic strokes.
- CyberDildonics 11y ago
  What does the GPU accelerate? I actually pretty surprised that there is anything that can be parallelized to that level. Edit: signed distance field font rendering? Nice
  - pcwalton 11y ago
    > What does the GPU accelerate? I actually pretty surprised that there is anything that can be parallelized to that level. Essentially everything but font rasterization (assuming you don't count image decode as part of painting). The major trick here is to avoid vector graphics, since you don't need a full-blown vector graphics library to paint common CSS. > signed distance field font rendering? Nice Note that this isn't integrated yet; it's a pile of experimental code, and I have some lingering doubts about its performance. But I would love it if we (or someone else!) can make it work!
    - zbraniecki 11y ago
      There's a lot of reasonable argument put behind the concept that we will use SVG more and more on the web. You're saying that WebRenderer should work for faster for anything but vector graphics. Is there a risk that it's optimizing for todays/yesterdays use cases?
      - pcwalton 11y ago
        Eh, I'm skeptical. :) IMO, SVG has serious ergonomic and performance issues that make it unlikely that we'll see a large-scale transition to it anytime soon. Not to mention that vector graphics are not really what you want for mobile apps: the proven framework is something like Core Animation, which CSS is much closer to than SVG. But I don't really want to debate this here, as it's somewhat off-topic. And there are only so many ways to draw a button, SVG or not. WebRender is really good at drawing buttons. So there's a possibility that we could use the same technique, just extended, for SVG.
- trishume 11y ago
  Do you have any good comparison numbers from Firefox or Chrome? The numbers sure seem impressive when compared to Servo but I know that Chrome has some pretty awesome rendering infrastructure and I'm wondering how this compares. There was one screenshot of Firefox included but it seemed to cherry pick the longest paint and didn't offer any explanation of how to compare it or which Servo+Webrender numbers to compare it to.
  - pcwalton 11y ago
    There have been some really back-of-the-envelope comparisons with Skia-GL, but nothing rigorous enough to write up. That's one major reason why the numbers should be taken with a grain of salt. Servo uses the same painting backend as Firefox does: Moz2D (also known as Azure). Moz2D is itself an abstraction layer over Cairo, Skia, Core Graphics, and Direct2D. So comparing against current Servo is actually a good way to compare against Firefox and Chrome in CPU painting mode: it's using the same painting backend as both of those browsers and eliminates any noise that results from layout differences. The most important thing to note is that the overall approach is quite different from Skia-GL (Ganesh). Skia is largely an immediate mode API (with a few retained-mode-style optimizations available with SkPictures, but the overall approach is immediate). WebRender (preliminary name), however, is a fully retained mode API and has no immediate mode at all, allowing it to focus on aggressive optimizations like "strength-reducing" clipping regions and Z-buffering opaque content for improved batching. This is a tradeoff, and it'd be easy to construct pathological cases in which one API or the other performs better. What we need to find out is which wins on real-world content, and while our early results are promising (and intuitively it seems highly probable that a retained mode API will win for CSS and the DOM), there's a lot more investigation that needs to happen before it'll be a slam-dunk case.
    - JoshTriplett 11y ago
      > Moz2D is itself an abstraction layer over Cairo, Skia, Core Graphics, and Direct2D. Out of curiosity, given that both Cairo and Skia run on all platforms Firefox does and have GPU-accelerated backends, why abstract over more than one of them rather than picking one and optimizing it for all platforms?
- bjwbell 11y ago
  It's amazing work. The big question is keep two renderers, CPU & GPU, or switch to just GPU? Switching to only GPU I think implies the need for at least a reference GPU backend such as mesa.
  - pcwalton 11y ago
    We're already OpenGL-dependent in the compositor. All this does is expand the set of GL features we use a bit.
    - bjwbell 11y ago
      I was thinking more about passing the reference test suites. Or in the case this case failing since GPUs won't all render identically.
      - pcwalton 11y ago
        We still need a headless GL compositor even today to pass the reftests. The only way to composite layers together is GL. (This will probably change when we support printing though.)
        JoshTriplett 11y ago
        > The only way to composite layers together is GL. (This will probably change when we support printing though.) If not for the (very reasonable) expectation that things like "print to PDF" should produce vectors and selectable text rather than raster images of pages, you could render pages for printing using GL, too.
        pcwalton 11y ago
        Yeah, I mainly mean PDF output here.
- dang 11y ago
  > The title on HN is very editorialized Indeed. Care to suggest one that's accurate and neutral? (In the meantime we'll take out the word "impressive".) Congratulations on the ongoing work, too!
  - pcwalton 11y ago
    Hmm, maybe something along the lines of "Servo WebRender overview". Thanks!
    - dang 11y ago
      Ok, we'll use that.
  - deleted 11y ago
    [deleted]
  - Flenser 11y ago
    Someone should propose a new element to the w3c <nonsensationaltitleforhn>
    - voltagex_ 11y ago
      <title data-hn="...">One Weird Trick for extra data in HTML tags</title>
      - Flenser 11y ago
        well sure, that would be the pragmatic way to do it, but it kinda spoils the joke.
  - wging 11y ago
    What was the title before it was changed? Right now it says " Servo WebRender Overview".
    - dang 11y ago
- mtgx 11y ago
  So will we see Servo in either the mobile browser or the desktop one next year?
modeless 11y ago
I'd love to see a Web renderer based on Vulkan. The reduced CPU overhead and much improved threading should be a huge win. You'd be able to cache on the command buffer level instead of just the vertex buffer level. Seems like a bleeding edge graphics API would be a good match for an experimental project like Servo as well. To really use Vulkan as efficiently as possible, though, you'd probably want to write a renderer mostly from scratch rather than maintaining some sort of abstraction layer to support both GL and Vulkan.
- pcwalton 11y ago
  > I'd love to see a Web renderer based on Vulkan. I agree! The current graphics and driver landscape is too fragmented to bet on Vulkan in the near-term, however. We do anticipate being able to keep most of the code around when Vulkan hits the scene. Our approach isn't to write an abstraction layer per se but rather by keeping our direct GL usage as isolated as possible, so that we can easily rewrite that stuff from scratch to really make good use of Vulkan, as you say. (This also allows us to support D3D if we want to.) Most of the logic here—batching, culling, texture atlas management, display list optimization—is very general logic that will apply no matter what hardware API we end up using.
  - modeless 11y ago
    My impression has been that Servo isn't focusing on the near-term. Is that changing?
    - pcwalton 11y ago
      "Near term" is relative. Multicore has been here for many years and we can safely assume its presence (though even then we're making sure we have good sequential performance). But it's not even clear that Vulkan will be available at all on all devices we might care about in the foreseeable future.
okasaki 11y ago
I often run browsers over ssh and I really hope this "draw programs like a game engine" stuff doesn't catch on. This is already the approach taken by Chrome (trace Chrome and you'll just see a bunch of full window resolution PutImage calls), and it's very slow over the network, I get around 1 fps. Firefox on the other hand uses more X11 calls, and it's much more responsive, close to running it locally.
- comex 11y ago
  I'd call this more a gap in modern remoting functionality on Linux than anything else. With the prevalence of hardware encoding and decoding, there is little reason not to just send H.264 (or your choice of video codec) over the network. Not (just) to compensate for the kind of issue you mention, but also because it allows animations, videos, games, etc. to work over the network, and is likely to compress better than X11 anyway. Actually, I believe NX already does this in the latest (proprietary) version, though I don't use it.
  - vidarh 11y ago
    XPRA supports various encodings like this: http://xpra.org/ http://xpra.org/ But video codecs does not give the same experience as running apps that are actually written to take some care about what/how it updates. Not least because the artefacts when they can't keep up are less consistent with what I at least expect from a desktop environment.
  - deleted 11y ago
    [deleted]
  - tedunangst 11y ago
    Apart from the fact that if nothing changes, it shouldn't send anything. I don't feel like burning 100k/s in data just to receive a pixelated image that doesn't change.
- pjmlp 11y ago
  RDP doesn't have any issue doing remote DirectX rendering. Just as mentioned in the sibling post, X11 isn't up the task for modern graphics. EDIT: lots of typos
devit 11y ago
Is there an explanation of how exactly this differs from current browsers? All things that are described in the page seem pretty "normal" and "obvious" things to do, so it's not quite clear which things are not being done in current browsers.
- pcwalton 11y ago
  The main differences, as I explained in another comment, are that (a) the API is fully retained-mode, not immediate-mode; (b) CSS is rendered directly without going through a vector graphics API first. The former feature allows us to achieve very aggressive batching and GL state optimizations, since we always know we have the entire draw list available to us. The latter feature has two benefits: (1) it makes all these optimizations much easier to do, since there are no vector paths involved and clipping regions are much simpler; (2) it allows us to skip all the difficult, slow stuff that arbitrary paths require (CPU path rendering, complex tessellation, stencil buffer clipping, etc.) in favor of simple hand-optimized code that renders CSS directly.
alex_duf 11y ago
I know servo and project around it are all experimental, but I can't help but feel excited about it.
mrmcq2u 11y ago
Great thing to see being worked on. Out of curiosity does any of Moz2D's back-ends support stencil then cover approach to rendering on the gpu. http://on-demand.gputechconf.com/gtc/2012/presentations/S0024-GPU-Accelerated-Path-Rendering.pdf http://on-demand.gputechconf.com/gtc/2012/presentations/S002... At first I thought it might be a good idea to implement a gallium statetracker for such a framework but with the direct3d statetracker not being supported by projects like wine I thought maybe vulkan will open up an opportunity to implement something with similar performance which is driver,vendor and platform agnostic.
- pcwalton 11y ago
  I think Skia-GL might (and who knows what Direct2D does). Note that one of the biggest benefits of WebRender is that it doesn't need to do any of that: if you render CSS directly, you can avoid all of these complex path rendering techniques.

191 points by dumindunuwan 11y ago

The title on HN is very editorialized and I can't overemphasize how preliminary and experimental this work is. Take all numbers with a huge grain of salt. That said, I'm obviously really excited about this project. :) I feel strongly that it's the right from-scratch, modern approach for painting Web content. Happy to answer any questions.

eridius 11y ago
Wow this looks very cool. Kudos!
- pcwalton 11y ago
  Glenn Watson deserves nearly all the credit here. My role has been limited to a little bit of high-level design input and messing around with signed distance fields and goofy stencil buffer tricks. :)
  - trishume 11y ago
    Ooooh signed distance fields, if you ever get a good Rust implementation going make sure to publish it and add a readme that at least says what the code is so that it can be found. It could be great for other rust libraries, apps and games. None of them have subpixel rendering at the moment AFAIK and that's super easy with distance fields and real important for appearance.
    - TD-Linux 11y ago
      >None of them have subpixel rendering at the moment AFAIK Actually, most Rust projects end up using Freetype for glyph rendering, which does support subpixel rendering. Signed distance fields are great for animated text, but they also have a couple of disadvantages - they generally produce lower quality output than Freetype at small glyph sizes, and the signed distance field is more complex to generate at start, which slows down initial page rendering. Given that 99% of web pages are static, it's a harder sell. Maybe some sort of heuristic based switching would work.
      - constexpr 11y ago
        I'm surprised to hear they produce lower quality output at small glyph sizes. Fragment shaders can do multiple texture samples and I've gotten very high quality results. You can even do ClearType-style subpixel rendering with a handful of samples. The case where signed distance fields don't work is for large glyph sizes. They often end up chopping off corners or not having the density to represent thin calligraphic strokes.
CyberDildonics 11y ago
What does the GPU accelerate? I actually pretty surprised that there is anything that can be parallelized to that level. Edit: signed distance field font rendering? Nice
- pcwalton 11y ago
  > What does the GPU accelerate? I actually pretty surprised that there is anything that can be parallelized to that level. Essentially everything but font rasterization (assuming you don't count image decode as part of painting). The major trick here is to avoid vector graphics, since you don't need a full-blown vector graphics library to paint common CSS. > signed distance field font rendering? Nice Note that this isn't integrated yet; it's a pile of experimental code, and I have some lingering doubts about its performance. But I would love it if we (or someone else!) can make it work!
  - zbraniecki 11y ago
    There's a lot of reasonable argument put behind the concept that we will use SVG more and more on the web. You're saying that WebRenderer should work for faster for anything but vector graphics. Is there a risk that it's optimizing for todays/yesterdays use cases?
    - pcwalton 11y ago
      Eh, I'm skeptical. :) IMO, SVG has serious ergonomic and performance issues that make it unlikely that we'll see a large-scale transition to it anytime soon. Not to mention that vector graphics are not really what you want for mobile apps: the proven framework is something like Core Animation, which CSS is much closer to than SVG. But I don't really want to debate this here, as it's somewhat off-topic. And there are only so many ways to draw a button, SVG or not. WebRender is really good at drawing buttons. So there's a possibility that we could use the same technique, just extended, for SVG.
trishume 11y ago
Do you have any good comparison numbers from Firefox or Chrome? The numbers sure seem impressive when compared to Servo but I know that Chrome has some pretty awesome rendering infrastructure and I'm wondering how this compares. There was one screenshot of Firefox included but it seemed to cherry pick the longest paint and didn't offer any explanation of how to compare it or which Servo+Webrender numbers to compare it to.
- pcwalton 11y ago
  There have been some really back-of-the-envelope comparisons with Skia-GL, but nothing rigorous enough to write up. That's one major reason why the numbers should be taken with a grain of salt. Servo uses the same painting backend as Firefox does: Moz2D (also known as Azure). Moz2D is itself an abstraction layer over Cairo, Skia, Core Graphics, and Direct2D. So comparing against current Servo is actually a good way to compare against Firefox and Chrome in CPU painting mode: it's using the same painting backend as both of those browsers and eliminates any noise that results from layout differences. The most important thing to note is that the overall approach is quite different from Skia-GL (Ganesh). Skia is largely an immediate mode API (with a few retained-mode-style optimizations available with SkPictures, but the overall approach is immediate). WebRender (preliminary name), however, is a fully retained mode API and has no immediate mode at all, allowing it to focus on aggressive optimizations like "strength-reducing" clipping regions and Z-buffering opaque content for improved batching. This is a tradeoff, and it'd be easy to construct pathological cases in which one API or the other performs better. What we need to find out is which wins on real-world content, and while our early results are promising (and intuitively it seems highly probable that a retained mode API will win for CSS and the DOM), there's a lot more investigation that needs to happen before it'll be a slam-dunk case.
  - JoshTriplett 11y ago
    > Moz2D is itself an abstraction layer over Cairo, Skia, Core Graphics, and Direct2D. Out of curiosity, given that both Cairo and Skia run on all platforms Firefox does and have GPU-accelerated backends, why abstract over more than one of them rather than picking one and optimizing it for all platforms?
bjwbell 11y ago
It's amazing work. The big question is keep two renderers, CPU & GPU, or switch to just GPU? Switching to only GPU I think implies the need for at least a reference GPU backend such as mesa.
- pcwalton 11y ago
  We're already OpenGL-dependent in the compositor. All this does is expand the set of GL features we use a bit.
  - bjwbell 11y ago
    I was thinking more about passing the reference test suites. Or in the case this case failing since GPUs won't all render identically.
    - pcwalton 11y ago
      We still need a headless GL compositor even today to pass the reftests. The only way to composite layers together is GL. (This will probably change when we support printing though.)
      - JoshTriplett 11y ago
        > The only way to composite layers together is GL. (This will probably change when we support printing though.) If not for the (very reasonable) expectation that things like "print to PDF" should produce vectors and selectable text rather than raster images of pages, you could render pages for printing using GL, too.
        pcwalton 11y ago
        Yeah, I mainly mean PDF output here.
dang 11y ago
> The title on HN is very editorialized Indeed. Care to suggest one that's accurate and neutral? (In the meantime we'll take out the word "impressive".) Congratulations on the ongoing work, too!
- pcwalton 11y ago
  Hmm, maybe something along the lines of "Servo WebRender overview". Thanks!
  - dang 11y ago
    Ok, we'll use that.
- deleted 11y ago
  [deleted]
- Flenser 11y ago
  Someone should propose a new element to the w3c <nonsensationaltitleforhn>
  - voltagex_ 11y ago
    <title data-hn="...">One Weird Trick for extra data in HTML tags</title>
    - Flenser 11y ago
      well sure, that would be the pragmatic way to do it, but it kinda spoils the joke.
- wging 11y ago
  What was the title before it was changed? Right now it says " Servo WebRender Overview".
  - dang 11y ago
mtgx 11y ago
So will we see Servo in either the mobile browser or the desktop one next year?

modeless 11y ago

I'd love to see a Web renderer based on Vulkan. The reduced CPU overhead and much improved threading should be a huge win. You'd be able to cache on the command buffer level instead of just the vertex buffer level. Seems like a bleeding edge graphics API would be a good match for an experimental project like Servo as well. To really use Vulkan as efficiently as possible, though, you'd probably want to write a renderer mostly from scratch rather than maintaining some sort of abstraction layer to support both GL and Vulkan.

pcwalton 11y ago
> I'd love to see a Web renderer based on Vulkan. I agree! The current graphics and driver landscape is too fragmented to bet on Vulkan in the near-term, however. We do anticipate being able to keep most of the code around when Vulkan hits the scene. Our approach isn't to write an abstraction layer per se but rather by keeping our direct GL usage as isolated as possible, so that we can easily rewrite that stuff from scratch to really make good use of Vulkan, as you say. (This also allows us to support D3D if we want to.) Most of the logic here—batching, culling, texture atlas management, display list optimization—is very general logic that will apply no matter what hardware API we end up using.
- modeless 11y ago
  My impression has been that Servo isn't focusing on the near-term. Is that changing?
  - pcwalton 11y ago
    "Near term" is relative. Multicore has been here for many years and we can safely assume its presence (though even then we're making sure we have good sequential performance). But it's not even clear that Vulkan will be available at all on all devices we might care about in the foreseeable future.

okasaki 11y ago

I often run browsers over ssh and I really hope this "draw programs like a game engine" stuff doesn't catch on. This is already the approach taken by Chrome (trace Chrome and you'll just see a bunch of full window resolution PutImage calls), and it's very slow over the network, I get around 1 fps. Firefox on the other hand uses more X11 calls, and it's much more responsive, close to running it locally.

comex 11y ago
I'd call this more a gap in modern remoting functionality on Linux than anything else. With the prevalence of hardware encoding and decoding, there is little reason not to just send H.264 (or your choice of video codec) over the network. Not (just) to compensate for the kind of issue you mention, but also because it allows animations, videos, games, etc. to work over the network, and is likely to compress better than X11 anyway. Actually, I believe NX already does this in the latest (proprietary) version, though I don't use it.
- vidarh 11y ago
  XPRA supports various encodings like this: http://xpra.org/ http://xpra.org/ But video codecs does not give the same experience as running apps that are actually written to take some care about what/how it updates. Not least because the artefacts when they can't keep up are less consistent with what I at least expect from a desktop environment.
- deleted 11y ago
  [deleted]
- tedunangst 11y ago
  Apart from the fact that if nothing changes, it shouldn't send anything. I don't feel like burning 100k/s in data just to receive a pixelated image that doesn't change.
pjmlp 11y ago
RDP doesn't have any issue doing remote DirectX rendering. Just as mentioned in the sibling post, X11 isn't up the task for modern graphics. EDIT: lots of typos

devit 11y ago

Is there an explanation of how exactly this differs from current browsers? All things that are described in the page seem pretty "normal" and "obvious" things to do, so it's not quite clear which things are not being done in current browsers.

pcwalton 11y ago
The main differences, as I explained in another comment, are that (a) the API is fully retained-mode, not immediate-mode; (b) CSS is rendered directly without going through a vector graphics API first. The former feature allows us to achieve very aggressive batching and GL state optimizations, since we always know we have the entire draw list available to us. The latter feature has two benefits: (1) it makes all these optimizations much easier to do, since there are no vector paths involved and clipping regions are much simpler; (2) it allows us to skip all the difficult, slow stuff that arbitrary paths require (CPU path rendering, complex tessellation, stencil buffer clipping, etc.) in favor of simple hand-optimized code that renders CSS directly.

alex_duf 11y ago

I know servo and project around it are all experimental, but I can't help but feel excited about it.

mrmcq2u 11y ago

Great thing to see being worked on. Out of curiosity does any of Moz2D's back-ends support stencil then cover approach to rendering on the gpu. http://on-demand.gputechconf.com/gtc/2012/presentations/S0024-GPU-Accelerated-Path-Rendering.pdf http://on-demand.gputechconf.com/gtc/2012/presentations/S002... At first I thought it might be a good idea to implement a gallium statetracker for such a framework but with the direct3d statetracker not being supported by projects like wine I thought maybe vulkan will open up an opportunity to implement something with similar performance which is driver,vendor and platform agnostic.

pcwalton 11y ago
I think Skia-GL might (and who knows what Direct2D does). Note that one of the biggest benefits of WebRender is that it doesn't need to do any of that: if you render CSS directly, you can avoid all of these complex path rendering techniques.