4 ms·

Woah this is incredible. Even with the increased CPU demands, this is a very large bandwidth savings. >In practice however, DRM, streaming , analytics and ad p

Woah this is incredible. Even with the increased CPU demands, this is a very large bandwidth savings. >In practice however, DRM, streaming , analytics and ad placement also require javascript logic to function in web runtimes, so in real word settings web-video playback can and does use a non-trivial amount of CPU time. I'm a little skeptical of this claim. No idea how much CPU is used for DRM, but I can't imagine it's on the order of multiple percentage points.

chmod775 6y ago
> Woah this is incredible. Even with the increased CPU demands, this is a very large bandwidth savings. GPUs happen to be very good at drawing shapes. So there's probably a lot of room for improvement, probably even enough that these kinds of videos could be faster than decoding "normal" video (for which modern CPUs and GPUs have dedicated silicon). This doesn't mean that I'm not entirely skeptical of this, especially because this probably will fail spectacularly for animation that isn't just vector graphics, which is most animation.
- kevingadd 6y ago
  GPUs are good at drawing shapes, but that doesn't actually mean that using the GPU to rasterize high-quality polygons will be fast. In practice most production solutions for this run on a combination of CPU and GPU and use complex shaders to achieve acceptable quality. If you're rendering regular alpha-blended or opaque triangles that's very fast, but they're not going to be anti-aliased and stuff like cubic splines will need to be triangulated on the CPU or GPU. Libraries like Pathfinder and Slug are the current state of the art and they are very complicated. In comparison, video acceleration on most modern GPUs is done using dedicated silicon that typically has its own performance budget (so it doesn't eat into the GPU time to run shaders or your compositor).
  - chrismorgan 6y ago
    Pathfinder and its ilk would breeze through this sort of thing, and are still improving constantly too, because so comparatively little work has been done in the space. Guessing very roughly based on the figures in https://raphlinus.github.io/rust/graphics/gpu/2020/06/13/fast-2d-rendering.html https://raphlinus.github.io/rust/graphics/gpu/2020/06/13/fas... (and Pathfinder is integrating some of the techniques there), I’d estimate that piet-gpu and soon Pathfinder, running on Intel integrated graphics, could comfortably draw each frame of this Simpsons sample video in well under one millisecond (for an effective maximum of 6% GPU/CPU usage), probably even under a tenth of a millisecond (0.6%). (Admittedly you could easily create a more complicated video which would be much more draining, e.g. paris-30k frames would struggle to hit 120fps on integrated graphics. Generally speaking it’s easier to make pathological cases in vector encodings than raster. But let’s forget about that.) At that point, the hardware separation of video decoder and other graphics stuff (which is a strong point in general, because it’s so computationally expensive) is much less important. You speak of Pathfinder and Slug as being “very complicated”, and indeed they are; but video decoding is hardly any less complicated, and much more computationally expensive.
    - kevingadd 6y ago
      Video decoding is computationally expensive, but it's done by dedicated silicon on most modern GPUs in order to deliver better battery life and better performance. So is encoding. Dedicated silicon is what these other techniques have to compete with - certainly possible, but not easy.
      - chrismorgan 6y ago
        It’s done in dedicated silicon in order to deliver better battery life and better performance, given how extremely computationally expensive it is. Playing back vector video is simply much less expensive, as it’s doing much less work. Assuming content like the sample videos here, I would expect vector video to be immediately competitive with raster video codecs in power efficiency and performance, once you shift the bulk of the vector rendering from CPU to GPU.
    - Jasper_ 6y ago
      Raph's technique relies on compute shaders as a core part of the processing technique. That wouldn't work on an overwhelming majority of phones these days. Pathfinder has a compute-less approach which moves the binning to the CPU, not the GPU. And we're talking about bandwidth-constrained devices. I don't think you're going to out-perform H264. Slug is not applicable to this scene -- it makes several core operating assumptions (being able to pre-process curve data into a friendlier format, and its shape bounding boxes are well-defined and pre-processed. These hold for font / text assets but not real-time glyphs)
b0rsuk 6y ago
Is this a net positive or a net negative as far as carbon emission go? With traditional techniques, the video can be recorded once and played "anywhere", no matter the level of detail. If you push the effort to the viewers, won't that increase overall power consumption? Nationalization of costs, privatization of profits.
IshKebab 6y ago
> this is a very large bandwidth savings. I'm going to need to see some evidence of that! I'm pretty sure H.265 or AV1 could encoder better quality videos at that bitrate.

chmod775 6y ago

> Woah this is incredible. Even with the increased CPU demands, this is a very large bandwidth savings. GPUs happen to be very good at drawing shapes. So there's probably a lot of room for improvement, probably even enough that these kinds of videos could be faster than decoding "normal" video (for which modern CPUs and GPUs have dedicated silicon). This doesn't mean that I'm not entirely skeptical of this, especially because this probably will fail spectacularly for animation that isn't just vector graphics, which is most animation.

kevingadd 6y ago
GPUs are good at drawing shapes, but that doesn't actually mean that using the GPU to rasterize high-quality polygons will be fast. In practice most production solutions for this run on a combination of CPU and GPU and use complex shaders to achieve acceptable quality. If you're rendering regular alpha-blended or opaque triangles that's very fast, but they're not going to be anti-aliased and stuff like cubic splines will need to be triangulated on the CPU or GPU. Libraries like Pathfinder and Slug are the current state of the art and they are very complicated. In comparison, video acceleration on most modern GPUs is done using dedicated silicon that typically has its own performance budget (so it doesn't eat into the GPU time to run shaders or your compositor).
- chrismorgan 6y ago
  Pathfinder and its ilk would breeze through this sort of thing, and are still improving constantly too, because so comparatively little work has been done in the space. Guessing very roughly based on the figures in https://raphlinus.github.io/rust/graphics/gpu/2020/06/13/fast-2d-rendering.html https://raphlinus.github.io/rust/graphics/gpu/2020/06/13/fas... (and Pathfinder is integrating some of the techniques there), I’d estimate that piet-gpu and soon Pathfinder, running on Intel integrated graphics, could comfortably draw each frame of this Simpsons sample video in well under one millisecond (for an effective maximum of 6% GPU/CPU usage), probably even under a tenth of a millisecond (0.6%). (Admittedly you could easily create a more complicated video which would be much more draining, e.g. paris-30k frames would struggle to hit 120fps on integrated graphics. Generally speaking it’s easier to make pathological cases in vector encodings than raster. But let’s forget about that.) At that point, the hardware separation of video decoder and other graphics stuff (which is a strong point in general, because it’s so computationally expensive) is much less important. You speak of Pathfinder and Slug as being “very complicated”, and indeed they are; but video decoding is hardly any less complicated, and much more computationally expensive.
  - kevingadd 6y ago
    Video decoding is computationally expensive, but it's done by dedicated silicon on most modern GPUs in order to deliver better battery life and better performance. So is encoding. Dedicated silicon is what these other techniques have to compete with - certainly possible, but not easy.
    - chrismorgan 6y ago
      It’s done in dedicated silicon in order to deliver better battery life and better performance, given how extremely computationally expensive it is. Playing back vector video is simply much less expensive, as it’s doing much less work. Assuming content like the sample videos here, I would expect vector video to be immediately competitive with raster video codecs in power efficiency and performance, once you shift the bulk of the vector rendering from CPU to GPU.
  - Jasper_ 6y ago
    Raph's technique relies on compute shaders as a core part of the processing technique. That wouldn't work on an overwhelming majority of phones these days. Pathfinder has a compute-less approach which moves the binning to the CPU, not the GPU. And we're talking about bandwidth-constrained devices. I don't think you're going to out-perform H264. Slug is not applicable to this scene -- it makes several core operating assumptions (being able to pre-process curve data into a friendlier format, and its shape bounding boxes are well-defined and pre-processed. These hold for font / text assets but not real-time glyphs)

b0rsuk 6y ago

Is this a net positive or a net negative as far as carbon emission go? With traditional techniques, the video can be recorded once and played "anywhere", no matter the level of detail. If you push the effort to the viewers, won't that increase overall power consumption? Nationalization of costs, privatization of profits.

IshKebab 6y ago

> this is a very large bandwidth savings. I'm going to need to see some evidence of that! I'm pretty sure H.265 or AV1 could encoder better quality videos at that bitrate.