7 ms·

Everyone is commenting on the product but what I want to know is how on earth they are able to encode AV1 in real time. I tried AV1 encoding a few months ago an

Everyone is commenting on the product but what I want to know is how on earth they are able to encode AV1 in real time. I tried AV1 encoding a few months ago and was getting about 0.3fps encoding 1080p video on an i7 4770k. I'm guessing the default settings on the av1 encoder are more focused on size over speed but still, how could they be getting 30fps video encoded in real time on a phone..

kabes 7y ago
Codecs usually have different modes. Typically a realtime mode trades quality for speed, e.g it will be just 1 pass with a lot of optimizations disabled.
plorkyeran 7y ago
I'm going to boldly guess that you were encoding at a significantly higher bitrate than the 30kbps they mention on the page. The lossless bitstream compression part of video encoding can be a very significant portion of the encoding time on faster speed settings, so lower bitrate means faster encodes. It also probably isn't 30 fps.
- emilfihlman 7y ago
  It's probably variable framerate and localised frame updates.
- Klinky 7y ago
  That demo looks too good for 30kbps for any codec, especially single pass real time. I am skeptical. Also find it funny they use a 2.5MB GIF to show a demo for ultra video compression. https://storage.googleapis.com/gweb-uniblog-publish-prod/original_images/Duo_540X540_TransparentBG_3s_splitscreen_05.gif https://storage.googleapis.com/gweb-uniblog-publish-prod/ori...
  - tambourine_man 7y ago
    We are discussing state of the art video codecs on a page that's serving 26MB GIFs.
    - ArnoVW 7y ago
      If you want to discuss the quality of video it is important that your own encoding does not alter the image. GIF allows animation and lossless compression.
      - Klinky 7y ago
        GIF only allows 256 colors per frame, so not really lossless when it comes to photos/video that exceed that limit. I guess there are some hacks around this, but typically they're not used.
        Strom 6y ago
        The hacks you allude to are based on the fact that GIF also allows you to use: * Transparency in a frame, which will show the previous frame underneath * A different color palette per frame * An infinite frame rate (no delay between two frames), which is also configured per frame These three points combined allow you to show any video in full 8-bit color via the GIF format. Unfortunately there's a difference between what GIF allows and what popular browsers allow GIF to do. Specifically all browsers set a minimum frame delay to 10ms == 100fps. This got started in the 90s due to performance concerns in Internet Explorer. Then people kept setting frame delay to 0 in their GIFs but it looked still fine in Internet Explorer. This resulted in a lot of GIFs that would look broken when the browsers would follow the GIF spec. This desire to support broken GIF files is why even in 2020 browsers limit GIF frame delay to 10ms. Even so, if you only need 25fps for your video, you can get 4*256=1024 colors. Again, these colors can be different from frame to frame.
    - naasking 7y ago
      What other lossless "video" format is widely available in browsers?
  - gorkish 7y ago
    I watched my fair share of RealPlayer video at about that bitrate 25 years ago. I see absolutely no reason that a modern codec designed for low bitrates couldn’t look that good on a talking head. It’s not like they are pandering though. They released the software. There is no need to speculate.
    - Klinky 7y ago
      The software is released, but I would like to see the actual encode instead of the phony looking side-by-side GIF.
  - asiachick 7y ago
    I'm just going to guess that's for those browsers that don't yet support AV1? I mean it basically just shipped https://caniuse.com/#feat=av1 https://caniuse.com/#feat=av1
    - Klinky 7y ago
      Chrome 81, still get GIFs.
motbob 7y ago
As someone who has been reading a lot of AV1 benchmarks over the last few months, it's also unclear to me how/why they're doing it. AV1 (and specifically google's libaom) is great, and for VOD-based content it is ready for primetime, but h264 is better for realtime streaming from just about every standpoint. Though, AV1 is way, way better than h264 when bitrates are super low... I don't know. I wish I had more details about the implementation.
- jplayer01 7y ago
  How low is super low and what's the picture quality like? Could it be they're trying to save bandwidth?
  - motbob 7y ago
    The best definition for "super low" that I can come up with is "bitrates at which x264-encoded content becomes hilariously unwatchable." I will try to be more specific. Youtube uses x264 to encode videos at a certain bitrate. Were you to encode the same video with libaom (AV1) at the same bitrate, the quality difference would probably be noticeable to a casual viewer. If you were to cut the bitrate in half and run the same comparison, the quality difference would be extremely noticeable to a casual viewer.
    - ehsankia 7y ago
      The article has a gif showing this. The right side is hilariously blocky, while the left is decent. Visually, from experience, it looks like somewhere between 100 to 200 kbps. EDIT: Err, I'm stupid it says 30kbps right there. Though that definitely looks closer to 100 to me (unless framerate is lower)
- aikinai 7y ago
  As someone up-to-date on AV1, what are your thoughts on it compared to HEVC?
  - motbob 7y ago
Jugurtha 7y ago
What I want to know is how on earth did they make the page 58.7 MB. This can't be.
- jml7c5 7y ago
  I thought that had to be a typo. Good lord, why are they shipping videos as gifs?
  - jtvjan 7y ago
    If it's stupid but it works, it's still stupid.
    - ithkuil 7y ago
      But it works. If it's clever and it doesn't work, it's surely clever ... but it doesn't work
      - emilfihlman 7y ago
        It has been known and well understood for many years now that you should use video (like mp4 or webm) in place of gifs if you want to save bandwidth.
        ithkuil 7y ago
        would encoding your AV1 encoding demo with another encode produce the desired effects?
  - kennydude 7y ago
    I think their blogs are still based off Blogger which hasn't been updated in quite some time. (which they own so they could fix themsleves..)
  - usrusr 7y ago
    There are only two ways of realistically demonstrating encoding artifacts: decode the actual format client side or transmit a lossless encoding. GIF surely isn't lossless, but its losses are somewhat orthogonal to those of the encodings in question. Still, yet could have done with much less of the same.
sp332 7y ago
The speed of an encoder really comes down to how many optimizations it's searching through. The AV1 format allows a lot more options when encoding, so encoder writers kinda feel obligated to try a bunch of them out to see if they save bandwidth. But if instead a codec just picked a few optimizations and ignored the rest, it could be pretty fast. Maybe Google just found a set that worked well for an average video chat.
- ehsankia 7y ago
  When streaming with x264 for example, most people usually use the "fastest" or "fast" preset, anything lower and you'll start dropping frames. Similarly I'm sure they can adjust levers to make av1 work in real time.
jupp0r 7y ago
I suspect that hardware encoding is involved. Maybe upcoming pixel phones will feature a previously unknown hardware encoder? It would make lots of sense, especially for storing videos made with the camera.
clouddrover 7y ago
> what I want to know is how on earth they are able to encode AV1 in real time Cisco also has a real time AV1 encoder. They say it's all about using the right AV1 video coding tools in the right way for the live video use case: https://blogs.cisco.com/collaboration/cisco-leap-frogs-h-264-video-collaboration-with-real-time-av1-codec https://blogs.cisco.com/collaboration/cisco-leap-frogs-h-264... The presentation: https://vimeo.com/344366650 https://vimeo.com/344366650
- jupp0r 7y ago
  That was on laptop cpus though. Even if Google optimized the encoder to accomplish this on mobile processors, a fully utilized cpu would drain batteries so fast that the user experience would suffer.
gergely 7y ago
not just that but if you consider that the stream is encrypted - so not just going through https - then it might be also great tech.

kabes 7y ago

Codecs usually have different modes. Typically a realtime mode trades quality for speed, e.g it will be just 1 pass with a lot of optimizations disabled.

plorkyeran 7y ago

I'm going to boldly guess that you were encoding at a significantly higher bitrate than the 30kbps they mention on the page. The lossless bitstream compression part of video encoding can be a very significant portion of the encoding time on faster speed settings, so lower bitrate means faster encodes. It also probably isn't 30 fps.

emilfihlman 7y ago
It's probably variable framerate and localised frame updates.
Klinky 7y ago
That demo looks too good for 30kbps for any codec, especially single pass real time. I am skeptical. Also find it funny they use a 2.5MB GIF to show a demo for ultra video compression. https://storage.googleapis.com/gweb-uniblog-publish-prod/original_images/Duo_540X540_TransparentBG_3s_splitscreen_05.gif https://storage.googleapis.com/gweb-uniblog-publish-prod/ori...
- tambourine_man 7y ago
  We are discussing state of the art video codecs on a page that's serving 26MB GIFs.
  - ArnoVW 7y ago
    If you want to discuss the quality of video it is important that your own encoding does not alter the image. GIF allows animation and lossless compression.
    - Klinky 7y ago
      GIF only allows 256 colors per frame, so not really lossless when it comes to photos/video that exceed that limit. I guess there are some hacks around this, but typically they're not used.
      - Strom 6y ago
        The hacks you allude to are based on the fact that GIF also allows you to use: * Transparency in a frame, which will show the previous frame underneath * A different color palette per frame * An infinite frame rate (no delay between two frames), which is also configured per frame These three points combined allow you to show any video in full 8-bit color via the GIF format. Unfortunately there's a difference between what GIF allows and what popular browsers allow GIF to do. Specifically all browsers set a minimum frame delay to 10ms == 100fps. This got started in the 90s due to performance concerns in Internet Explorer. Then people kept setting frame delay to 0 in their GIFs but it looked still fine in Internet Explorer. This resulted in a lot of GIFs that would look broken when the browsers would follow the GIF spec. This desire to support broken GIF files is why even in 2020 browsers limit GIF frame delay to 10ms. Even so, if you only need 25fps for your video, you can get 4*256=1024 colors. Again, these colors can be different from frame to frame.
  - naasking 7y ago
    What other lossless "video" format is widely available in browsers?
- gorkish 7y ago
  I watched my fair share of RealPlayer video at about that bitrate 25 years ago. I see absolutely no reason that a modern codec designed for low bitrates couldn’t look that good on a talking head. It’s not like they are pandering though. They released the software. There is no need to speculate.
  - Klinky 7y ago
    The software is released, but I would like to see the actual encode instead of the phony looking side-by-side GIF.
- asiachick 7y ago
  I'm just going to guess that's for those browsers that don't yet support AV1? I mean it basically just shipped https://caniuse.com/#feat=av1 https://caniuse.com/#feat=av1
  - Klinky 7y ago
    Chrome 81, still get GIFs.

motbob 7y ago

As someone who has been reading a lot of AV1 benchmarks over the last few months, it's also unclear to me how/why they're doing it. AV1 (and specifically google's libaom) is great, and for VOD-based content it is ready for primetime, but h264 is better for realtime streaming from just about every standpoint. Though, AV1 is way, way better than h264 when bitrates are super low... I don't know. I wish I had more details about the implementation.

jplayer01 7y ago
How low is super low and what's the picture quality like? Could it be they're trying to save bandwidth?
- motbob 7y ago
  The best definition for "super low" that I can come up with is "bitrates at which x264-encoded content becomes hilariously unwatchable." I will try to be more specific. Youtube uses x264 to encode videos at a certain bitrate. Were you to encode the same video with libaom (AV1) at the same bitrate, the quality difference would probably be noticeable to a casual viewer. If you were to cut the bitrate in half and run the same comparison, the quality difference would be extremely noticeable to a casual viewer.
  - ehsankia 7y ago
    The article has a gif showing this. The right side is hilariously blocky, while the left is decent. Visually, from experience, it looks like somewhere between 100 to 200 kbps. EDIT: Err, I'm stupid it says 30kbps right there. Though that definitely looks closer to 100 to me (unless framerate is lower)
aikinai 7y ago
As someone up-to-date on AV1, what are your thoughts on it compared to HEVC?
- motbob 7y ago

Jugurtha 7y ago

What I want to know is how on earth did they make the page 58.7 MB. This can't be.

jml7c5 7y ago
I thought that had to be a typo. Good lord, why are they shipping videos as gifs?
- jtvjan 7y ago
  If it's stupid but it works, it's still stupid.
  - ithkuil 7y ago
    But it works. If it's clever and it doesn't work, it's surely clever ... but it doesn't work
    - emilfihlman 7y ago
      It has been known and well understood for many years now that you should use video (like mp4 or webm) in place of gifs if you want to save bandwidth.
      - ithkuil 7y ago
        would encoding your AV1 encoding demo with another encode produce the desired effects?
- kennydude 7y ago
  I think their blogs are still based off Blogger which hasn't been updated in quite some time. (which they own so they could fix themsleves..)
- usrusr 7y ago
  There are only two ways of realistically demonstrating encoding artifacts: decode the actual format client side or transmit a lossless encoding. GIF surely isn't lossless, but its losses are somewhat orthogonal to those of the encodings in question. Still, yet could have done with much less of the same.

sp332 7y ago

The speed of an encoder really comes down to how many optimizations it's searching through. The AV1 format allows a lot more options when encoding, so encoder writers kinda feel obligated to try a bunch of them out to see if they save bandwidth. But if instead a codec just picked a few optimizations and ignored the rest, it could be pretty fast. Maybe Google just found a set that worked well for an average video chat.

ehsankia 7y ago
When streaming with x264 for example, most people usually use the "fastest" or "fast" preset, anything lower and you'll start dropping frames. Similarly I'm sure they can adjust levers to make av1 work in real time.

jupp0r 7y ago

I suspect that hardware encoding is involved. Maybe upcoming pixel phones will feature a previously unknown hardware encoder? It would make lots of sense, especially for storing videos made with the camera.

clouddrover 7y ago

> what I want to know is how on earth they are able to encode AV1 in real time Cisco also has a real time AV1 encoder. They say it's all about using the right AV1 video coding tools in the right way for the live video use case: https://blogs.cisco.com/collaboration/cisco-leap-frogs-h-264-video-collaboration-with-real-time-av1-codec https://blogs.cisco.com/collaboration/cisco-leap-frogs-h-264... The presentation: https://vimeo.com/344366650 https://vimeo.com/344366650

jupp0r 7y ago
That was on laptop cpus though. Even if Google optimized the encoder to accomplish this on mobile processors, a fully utilized cpu would drain batteries so fast that the user experience would suffer.

gergely 7y ago

not just that but if you consider that the stream is encrypted - so not just going through https - then it might be also great tech.