6 ms·

This guide will be relevant until OS compositors fundamentally change from just dealing with bitmaps. Which is unlikely to happen anytime soon. And doing UI la

This guide will be relevant until OS compositors fundamentally change from just dealing with bitmaps. Which is unlikely to happen anytime soon. And doing UI layout on background threads breaks the basic design of pretty much every UI framework, web or native, which are usually single-threaded.

westoncb 9y ago
Hmm... Wouldn't it be possible to address the problem at another level? For instance, when it's discovered that layout properties are being animated, see if it would be possible to create an equivalent effect in the compositing stage, if so, convert the layout position changes to transforms? I realize that would probably not be easy, but maybe still possible without major architectural changes?
- whowouldathunk 9y ago
  Which thread are you going to do that work on? It's complicated because layout changes can cause new elements to be created. Elements can be really expensive to create.
  - westoncb 9y ago
    That's the kind of thing that would cause a rejection: elements are gong to be created, so this can't be converted to compositing stage operations instead. Dunno which thread—just an idea.
  - pcwalton 9y ago
    > Which thread are you going to do that work on? The layout thread. > It's complicated because layout changes can cause new elements to be created. Elements can be really expensive to create. Huh? No, they can't. In rare cases, layout changes can cause new render objects to be created (line breaks), but that's not elements.
    - whowouldathunk 9y ago
      Not sure how Servo works, but what happens if a list viewport gets bigger and you need to display more list items?
      - pcwalton 9y ago
        The newly visible list items are already laid out, so we just display them.
- pcwalton 9y ago
  > For instance, when it's discovered that layout properties are being animated, see if it would be possible to create an equivalent effect in the compositing stage, if so, convert the layout position changes to transforms? Yes, and we as browser vendors should absolutely do that. Though a better solution is to just make the compositor able to accelerate the full generality of CSS in the first place.
ZenoArrow 9y ago
> "Which is unlikely to happen anytime soon." Mozilla are moving Servo's WebRender over to Firefox. The feature is called Quantum Render, you can read up about it here: https://wiki.mozilla.org/Platform/GFX/Quantum_Render https://wiki.mozilla.org/Platform/GFX/Quantum_Render To give some idea of what this means for CSS animation performance: https://m.youtube.com/watch?v=u0hYIRQRiws https://m.youtube.com/watch?v=u0hYIRQRiws
kllrnohj 9y ago
That's not true at all. You don't have most of these problems when dealing with native apps, for example, and the reason is that browser rendering is just extremely slow. Thus the gap between the common path and the fast path is unusually massive on browsers. It will get better when things like this: https://bugs.chromium.org/p/chromium/issues/detail?id=591179#c16 https://bugs.chromium.org/p/chromium/issues/detail?id=591179... get marked fixed (aka, use GPU rendering everywhere) Although browsers needing to be highly defensive doesn't help, either. If a native app renders slow you generally blame the app, whereas if a website is slow you blame the browser (regardless of who is actually at fault). This leads to browser being conservative and defensive in their graphics stacks so that scrolling can be smooth in the face of graphically intense rendering. OS composition is unrelated entirely here, and really there's no need for that to change away from just dealing with pixmaps.
- whowouldathunk 9y ago
  You have this problem with native Windows apps. I'm not familiar with iOS or Android. https://blogs.msdn.microsoft.com/windowsappdev/2012/05/01/fast-and-fluid-animations-in-your-metro-style-app/ https://blogs.msdn.microsoft.com/windowsappdev/2012/05/01/fa... An independent animation is an animation that runs independently from thread running the core UI logic. (A dependent animation runs on the UI thread.) The elements map directly to OS compositor "visuals." Disclosure: I work at Microsoft
  - kuschku 9y ago
    The same issues also happen on Android, which is why Android has separate threads for layouting and interactions, rendering, and business logic in every app, and why all complicated rendering ends up on separate GPU layers.
    - kllrnohj 9y ago
      > and why all complicated rendering ends up on separate GPU layers. No, it doesn't. The app renders to a single surface in a single GPU render pass unless the app uses a SurfaceView, which is generally only for media uses (camera, video, games). Multiple layers are only used when asked for explicitly (View.setLayerType) or when required for proper blending. They are generally avoided otherwise as it's generally slower to use multiple layers. You can absolutely do the "bad" things in the linked article in a native Android app and still hit 60fps pretty trivially. The accelerated properties, like View.setTranslationX/Y, only bypass what's typically a small amount of work (and don't use a caching layer). It's an incremental improvement, not something absolutely required. Scrolling in a RecyclerView or ListView, for example, doesn't even do that. It just moves the left/top of every view and re-renders, and that's plenty fast to hit 60fps.
pcwalton 9y ago
> This guide will be relevant until OS compositors fundamentally change from just dealing with bitmaps. Which is unlikely to happen anytime soon. There is absolutely no reason why browsers have to use the native compositor for CSS. It's a bad fit, and browsers should stop doing it. > And doing UI layout on background threads breaks the basic design of pretty much every UI framework, web or native, which are usually single-threaded. That's why you have a render tree on a separate thread from the DOM. Look at how Servo does it (disclaimer: I work on Servo). We've proven that it works.
- whowouldathunk 9y ago
  > There is absolutely no reason why browsers have to use the native compositor for CSS. Sure, but you're basically re-implementing big parts of the OS in the browser. Which is maybe not a bad idea, but at some point there's inception.
  - pcwalton 9y ago
    There's a good reason to, in this case: that the OS compositor is not designed to render the full generality of CSS. The status quo is not working: only about half of CSS animations in the wild run on the compositor. As browser vendors, we need to admit that the attempt to carve out a limited, fast subset of CSS has failed, and we should do the hard work to make all of CSS run fast.
- kllrnohj 9y ago
  > There is absolutely no reason why browsers have to use the native compositor for CSS. It's a bad fit, and browsers should stop doing it. Not entirely true. It's pretty critical to use the OS compositor for <video> for power reasons so that video playback can be offloaded entirely. For everything else, though, agree. The OS compositor isn't magic and at the end of the day doesn't do much for non-video layers.
- panic 9y ago
  > There is absolutely no reason why browsers have to use the native compositor for CSS. It's a bad fit, and browsers should stop doing it. Here are a few: - On some platforms, animations on native layers are applied every frame, even if the application that owns the layer is busy. This means fewer opportunities to drop frames. - Sometimes web rendering engines are embedded in apps. Your app may want to draw web content that filters some other content that's behind it. This content may not be available to the browser engine (it may be in a separate process, for example). A native layer can apply this filter in the compositor process, where the content is available. - Using the native compositor makes it easier to embed components (like video) that are provided by the system as native layers.
runeks 9y ago
> This guide will be relevant until OS compositors fundamentally change from just dealing with bitmaps. Which is unlikely to happen anytime soon. A browser is essentially a small OS with server-side rendering, where clients/web apps send HTML/CSS/JS for their GUI and the compositor/browser engine renders into bitmaps. We probably need something simpler than HTML/CSS/JS, though, if we want it to be reasonably easy to implement a fast rendering engine.

westoncb 9y ago

Hmm... Wouldn't it be possible to address the problem at another level? For instance, when it's discovered that layout properties are being animated, see if it would be possible to create an equivalent effect in the compositing stage, if so, convert the layout position changes to transforms? I realize that would probably not be easy, but maybe still possible without major architectural changes?

whowouldathunk 9y ago
Which thread are you going to do that work on? It's complicated because layout changes can cause new elements to be created. Elements can be really expensive to create.
- westoncb 9y ago
  That's the kind of thing that would cause a rejection: elements are gong to be created, so this can't be converted to compositing stage operations instead. Dunno which thread—just an idea.
- pcwalton 9y ago
  > Which thread are you going to do that work on? The layout thread. > It's complicated because layout changes can cause new elements to be created. Elements can be really expensive to create. Huh? No, they can't. In rare cases, layout changes can cause new render objects to be created (line breaks), but that's not elements.
  - whowouldathunk 9y ago
    Not sure how Servo works, but what happens if a list viewport gets bigger and you need to display more list items?
    - pcwalton 9y ago
      The newly visible list items are already laid out, so we just display them.
pcwalton 9y ago
> For instance, when it's discovered that layout properties are being animated, see if it would be possible to create an equivalent effect in the compositing stage, if so, convert the layout position changes to transforms? Yes, and we as browser vendors should absolutely do that. Though a better solution is to just make the compositor able to accelerate the full generality of CSS in the first place.

ZenoArrow 9y ago

> "Which is unlikely to happen anytime soon." Mozilla are moving Servo's WebRender over to Firefox. The feature is called Quantum Render, you can read up about it here: https://wiki.mozilla.org/Platform/GFX/Quantum_Render https://wiki.mozilla.org/Platform/GFX/Quantum_Render To give some idea of what this means for CSS animation performance: https://m.youtube.com/watch?v=u0hYIRQRiws https://m.youtube.com/watch?v=u0hYIRQRiws

kllrnohj 9y ago

That's not true at all. You don't have most of these problems when dealing with native apps, for example, and the reason is that browser rendering is just extremely slow. Thus the gap between the common path and the fast path is unusually massive on browsers. It will get better when things like this: https://bugs.chromium.org/p/chromium/issues/detail?id=591179#c16 https://bugs.chromium.org/p/chromium/issues/detail?id=591179... get marked fixed (aka, use GPU rendering everywhere) Although browsers needing to be highly defensive doesn't help, either. If a native app renders slow you generally blame the app, whereas if a website is slow you blame the browser (regardless of who is actually at fault). This leads to browser being conservative and defensive in their graphics stacks so that scrolling can be smooth in the face of graphically intense rendering. OS composition is unrelated entirely here, and really there's no need for that to change away from just dealing with pixmaps.

whowouldathunk 9y ago
You have this problem with native Windows apps. I'm not familiar with iOS or Android. https://blogs.msdn.microsoft.com/windowsappdev/2012/05/01/fast-and-fluid-animations-in-your-metro-style-app/ https://blogs.msdn.microsoft.com/windowsappdev/2012/05/01/fa... An independent animation is an animation that runs independently from thread running the core UI logic. (A dependent animation runs on the UI thread.) The elements map directly to OS compositor "visuals." Disclosure: I work at Microsoft
- kuschku 9y ago
  The same issues also happen on Android, which is why Android has separate threads for layouting and interactions, rendering, and business logic in every app, and why all complicated rendering ends up on separate GPU layers.
  - kllrnohj 9y ago
    > and why all complicated rendering ends up on separate GPU layers. No, it doesn't. The app renders to a single surface in a single GPU render pass unless the app uses a SurfaceView, which is generally only for media uses (camera, video, games). Multiple layers are only used when asked for explicitly (View.setLayerType) or when required for proper blending. They are generally avoided otherwise as it's generally slower to use multiple layers. You can absolutely do the "bad" things in the linked article in a native Android app and still hit 60fps pretty trivially. The accelerated properties, like View.setTranslationX/Y, only bypass what's typically a small amount of work (and don't use a caching layer). It's an incremental improvement, not something absolutely required. Scrolling in a RecyclerView or ListView, for example, doesn't even do that. It just moves the left/top of every view and re-renders, and that's plenty fast to hit 60fps.

pcwalton 9y ago

> This guide will be relevant until OS compositors fundamentally change from just dealing with bitmaps. Which is unlikely to happen anytime soon. There is absolutely no reason why browsers have to use the native compositor for CSS. It's a bad fit, and browsers should stop doing it. > And doing UI layout on background threads breaks the basic design of pretty much every UI framework, web or native, which are usually single-threaded. That's why you have a render tree on a separate thread from the DOM. Look at how Servo does it (disclaimer: I work on Servo). We've proven that it works.

whowouldathunk 9y ago
> There is absolutely no reason why browsers have to use the native compositor for CSS. Sure, but you're basically re-implementing big parts of the OS in the browser. Which is maybe not a bad idea, but at some point there's inception.
- pcwalton 9y ago
  There's a good reason to, in this case: that the OS compositor is not designed to render the full generality of CSS. The status quo is not working: only about half of CSS animations in the wild run on the compositor. As browser vendors, we need to admit that the attempt to carve out a limited, fast subset of CSS has failed, and we should do the hard work to make all of CSS run fast.
kllrnohj 9y ago
> There is absolutely no reason why browsers have to use the native compositor for CSS. It's a bad fit, and browsers should stop doing it. Not entirely true. It's pretty critical to use the OS compositor for <video> for power reasons so that video playback can be offloaded entirely. For everything else, though, agree. The OS compositor isn't magic and at the end of the day doesn't do much for non-video layers.
panic 9y ago
> There is absolutely no reason why browsers have to use the native compositor for CSS. It's a bad fit, and browsers should stop doing it. Here are a few: - On some platforms, animations on native layers are applied every frame, even if the application that owns the layer is busy. This means fewer opportunities to drop frames. - Sometimes web rendering engines are embedded in apps. Your app may want to draw web content that filters some other content that's behind it. This content may not be available to the browser engine (it may be in a separate process, for example). A native layer can apply this filter in the compositor process, where the content is available. - Using the native compositor makes it easier to embed components (like video) that are provided by the system as native layers.

runeks 9y ago

> This guide will be relevant until OS compositors fundamentally change from just dealing with bitmaps. Which is unlikely to happen anytime soon. A browser is essentially a small OS with server-side rendering, where clients/web apps send HTML/CSS/JS for their GUI and the compositor/browser engine renders into bitmaps. We probably need something simpler than HTML/CSS/JS, though, if we want it to be reasonably easy to implement a fast rendering engine.