4 ms·

The Last MPEG-4 Visual Patent Has Expired

225 points by LorenDB 2mo ago

p1mrx 2mo ago
Note that MPEG-4 part 2 is H.263/DivX/XviD, not H.264.
- jchw 2mo ago
  Yeah. I think h264 is coming soon, too, though. Most of the remaining known h264 patents expire next year, with only one patent extending out further. I do believe that more modern codecs like h265 and AV1 have a lot to offer us, but having h264 be fully unencumbered by patents will be nice. By modern standards it is very well-supported in both hardware and software, computationally cheap to encode and decode, and still does a good enough job for most use cases. h264 doesn't seem like it will be displaced any time soon to the degree that h264 once displaced its own predecessors, though obviously that's not to say that we don't see or won't see more adoption of newer video codecs across the internet in general, just that I think the h264 long tail will be one of the longest long tails of a video codec. And thanks to the availability of codecs like AV1 which are effectively not patent encumbered (seems there is no practical reason to take the threats otherwise seriously) it seems the era of patent encumbered media formats is slowly coming to a close. Good riddance. edit: Bit of a mess but here is some source for the h264 expiration. https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_MPEG-4_AVC_expired_yet%3F https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M... So it seems 2030-11-10 is the date where h264 (version 3) will become patent unencumbered world wide, a bit earlier for the U.S. The other profiles/newer versions like AVC/SVC will take a while longer.
  - Dylan16807 2mo ago
    > computationally cheap to encode and decode Well I'm not sure about decode, but as of a few years ago weren't h.265 and AV1 able to do much better compression than h.264 at almost all levels of compute? https://people.videolan.org/~unlord/SVT-AV1_BD-rate.png https://people.videolan.org/~unlord/SVT-AV1_BD-rate.png And h.266 encoders can probably go faster now too.
    - deleted 2mo ago
      [deleted]
    - jchw 2mo ago
      I would hope so! Otherwise the Pareto front wouldn't have moved in ~20 years which would be rather hard to believe. However, at the same time, h265, h266 and AV1 are all vastly more complicated, and will have less broad hardware acceleration support than h264, which, again, even if it is not optimally efficient use of compute, just doesn't require a whole lot by modern standards. So I'd argue there is really no reason to be rushing away from h264 unless you have a compelling reason. There are some obvious compelling reasons in some cases; nobody is going to be terribly surprised that an entity like Netflix is eager to switch to codecs that will save bandwidth, because at their scale saving bytes definitely adds up, bonus points if it can increase the quality at the same time. On the other hand, though, unless you are absolutely sure you can switch to only new codecs, you will probably want to keep some h264 encodings around to act as a fallback baseline for legacy devices. And thus, you also have to take into account the complexity brought on by needing to store and maintain multiple encodings - in many use cases, like simple <video>s thrown into website backgrounds, I can see just eating the extra bandwidth costs and keeping it all h264 as a valid strategy.
      - Dylan16807 2mo ago
        Well a lot of the improvements cost more computation. I wouldn't be surprised if low-computation video had only improved slightly, but it seems like it has improved a lot. The advantage of h264 is hardware adoption much more than actually being computationally cheap, it seems? Well, maybe. The thing is, I don't know how the decoders scale. Can you make a decoder for a constrained AV1/h.265 in the same hardware space or maximum CPU budget as an h.264 decoder and still see very big advantages?
      - duskwuff 2mo ago
        Once the patents expire, I expect to see H.264 become the JPEG of video codecs - it may not be the most efficient format available, but it's widely supported, relatively easy to encode/decode, and good enough quality for many use cases. (MP3 is in a similar place. AAC and Opus are technically superior, but everything supports MP3.)
  - userbinator 2mo ago
    just that I think the h264 long tail will be one of the longest long tails of a video codec. I think MPEG-1 and 2 (from which MP3 comes) might be even longer.
    - drmpeg 2mo ago
      MPEG-2 video is still very much alive. In the US, almost all terrestrial OTA broadcast (ATSC 1.0) and all SD channels on cable systems. Even Comcast VOD was all MPEG-2 (including HD) until they shut down the QAM version in 2023.
      - winstonwinston 2mo ago
        In Europe too, for Sd via DVB-C/T2. It is however non-existent for OTT.
      - jchw 2mo ago
        The way I see it, if you were to draw two graph lines showing the prevalence of H.262 and H.264 over time, it would show of course that H.262 has lasted much longer so far just by virtue of it being older yet still in active use, but the area under the H.264 line is probably already or tracking on being much larger, depending on many factors, such as how you quantify "prevalence". I don't see H.262 as becoming completely irrelevant any time soon, so it's clearly had a hell of a run, but I expect H.264 to wind up having a similarly long tail. I am not sure I'd bet on it being longer, but I wouldn't bet against it either. I'm only of the opinion that H.264 will have one of the longest of any video codec, but perhaps not the longest.
        drmpeg 2mo ago
        For sure. Just YouTube alone is a vast ocean of H.264 bits.
- userbinator 2mo ago
  DivX/XviD yes, but not H.263. Patents for H.263 expired long ago. This is MPEG-4 ASP, which is a superset of H.263. (Source: have written a H.263 decoder as a learning exercise.)
  - anthk 2mo ago
    H.263 was used as the former codec of Netmeeting (and maybe some other VoIP software too).
    - userbinator 2mo ago
      H.263 was definitely a standard in videoconferencing and low-bitrate video in general (YouTube and many other streaming sutes used FLV1, which was a variant of H.263) prior to H.264.
midnightbobarun 2mo ago
Very nice, I hope this means increased support for it in open source projects
theandrewbailey 2mo ago
> While the US and EU patents around MPEG-4 Part 2 expired in recent years, until today there remained one patent still active in Brazil. Since MPEG-this Part that is hard to keep straight, this is the MPEG standard backing Xvid and DivX codecs. So let's encode video and torrent like it's 2002!
- CharlesW 2mo ago
  FWIW, Via Licensing listed this patent as standards-essential to several MPEG codecs (AVC/H.264, HEVC/H.265, VVC/H.266), and it's also listed for VC-1 (a SMPTE standard).
  - p1mrx 2mo ago
    The openness of those codecs depend on the last patent expiring, not the first.
    - CharlesW 2mo ago
      Of course.
- dylan604 2mo ago
  > So let's encode video and torrent like it's 2002! Let's have a meet up a Circuit City!!
  - xattt 2mo ago
    I’m not sure how to get there. Do you have Microsoft Streets and Trips directions I can print off?
- ColdStream 2mo ago
  Time do get out the old CD of The.Matrix.1999-DVDrip(aXXo).Xvid.
  - anal_reactor 2mo ago
rcarmo 2mo ago
Good. I have been poking at rcarmo/go-264 but was hesitant to tackle encoding :)
rurban 2mo ago
But they sold xvid decoders already for the last 20 years. Every Chinese dvd players could play xvid already. Just not >1 b-frames
at1as 2mo ago
Moving in the right direction, but unfortunately it's still quite a few years before H264 is off patent, globally: https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_MPEG-4_AVC_expired_yet%3F https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M... At which point, the continued march towards higher resolution and bandwidth video will probably limit their utility
- ronsor 2mo ago
  > At which point, the continued march towards higher resolution and bandwidth video will probably limit their utility Just as the patent system intended: maximum extraction was enabled while the product was useful.
  - goalieca 2mo ago
    Now if only copyright didn't last forever..
    - ronsor 2mo ago
      The Berne Convention has permanently burned us there
      - account42 2mo ago
        Past international agreements are only as permanent as the people alive now agree they are. Yes this one will be very hard to change but that's just because there are many powerful interests that want to keep it this way.
    - snark_attack 2mo ago
      [dead]
    - cornell532 2mo ago
      Why? The Beatles really wrote and performed Strawberry Fields. Why are you entitled to it for gratis? It’s a taking for public benefit. Ancient writings like Plato or Marcus Aurelius somehow feel different. But high-fidelity live recording feels like it shouldn’t go into the public domain. It’s someone’s commercial work product
anthk 2mo ago
Too late; software patents in Europe were void so most people grabbed XVID/DIVX encoders from Debian Multimedia. PKF and the like. And, yes, I remember w32codecs until MPlayer's bundled set superseded them. And mencoder for sure ;)
nikanj 2mo ago
It's so weird to look at the patents list for H.264 and see multiple patents granted after the H.264 specification was released. Was the situation similar for MPEG-4?
- userbinator 2mo ago
  Those patents may not necessarily be required to encode or decode, if they're just a new way to implement some part of the specification.
- wmf 2mo ago
  The patent pool system is really bad. There's no incentive to weed out bad patents.
1317 2mo ago
as everyone is saying, this is Xvid, NOT h264. pretty poor reporting by phoronix here https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_MPEG-4_AVC_expired_yet%3F https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M...
aboardRat4 2mo ago
When MPEG-5?
- LtdJorge 2mo ago
  Never, I hope
- ksec 2mo ago
  There is. MPEG-5 EVC Basic Profile that is Patent unencumbered, with only dozens patents that is soon to be expired, and EVC High Profile that competes with VVC in terms of quality. There is also LCEVC ( Not connected with EVC ) that is a scaling layer on top of existing video codec. You can apply LCEVC on top of EVC, VVC, HEVC or AVC as well as AV1 and AV2. LCEVC has been selected by Brail as part of their TV 3.0 technology stack on top VVC. While there are momentum on LCEVC, I actually wished EVC gained more momentum. I guess everything is too late now.
hulitu 2mo ago
That's why they push for other codecs.

225 points by LorenDB 2mo ago

p1mrx 2mo ago

Note that MPEG-4 part 2 is H.263/DivX/XviD, not H.264.

jchw 2mo ago
Yeah. I think h264 is coming soon, too, though. Most of the remaining known h264 patents expire next year, with only one patent extending out further. I do believe that more modern codecs like h265 and AV1 have a lot to offer us, but having h264 be fully unencumbered by patents will be nice. By modern standards it is very well-supported in both hardware and software, computationally cheap to encode and decode, and still does a good enough job for most use cases. h264 doesn't seem like it will be displaced any time soon to the degree that h264 once displaced its own predecessors, though obviously that's not to say that we don't see or won't see more adoption of newer video codecs across the internet in general, just that I think the h264 long tail will be one of the longest long tails of a video codec. And thanks to the availability of codecs like AV1 which are effectively not patent encumbered (seems there is no practical reason to take the threats otherwise seriously) it seems the era of patent encumbered media formats is slowly coming to a close. Good riddance. edit: Bit of a mess but here is some source for the h264 expiration. https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_MPEG-4_AVC_expired_yet%3F https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M... So it seems 2030-11-10 is the date where h264 (version 3) will become patent unencumbered world wide, a bit earlier for the U.S. The other profiles/newer versions like AVC/SVC will take a while longer.
- Dylan16807 2mo ago
  > computationally cheap to encode and decode Well I'm not sure about decode, but as of a few years ago weren't h.265 and AV1 able to do much better compression than h.264 at almost all levels of compute? https://people.videolan.org/~unlord/SVT-AV1_BD-rate.png https://people.videolan.org/~unlord/SVT-AV1_BD-rate.png And h.266 encoders can probably go faster now too.
  - deleted 2mo ago
    [deleted]
  - jchw 2mo ago
    I would hope so! Otherwise the Pareto front wouldn't have moved in ~20 years which would be rather hard to believe. However, at the same time, h265, h266 and AV1 are all vastly more complicated, and will have less broad hardware acceleration support than h264, which, again, even if it is not optimally efficient use of compute, just doesn't require a whole lot by modern standards. So I'd argue there is really no reason to be rushing away from h264 unless you have a compelling reason. There are some obvious compelling reasons in some cases; nobody is going to be terribly surprised that an entity like Netflix is eager to switch to codecs that will save bandwidth, because at their scale saving bytes definitely adds up, bonus points if it can increase the quality at the same time. On the other hand, though, unless you are absolutely sure you can switch to only new codecs, you will probably want to keep some h264 encodings around to act as a fallback baseline for legacy devices. And thus, you also have to take into account the complexity brought on by needing to store and maintain multiple encodings - in many use cases, like simple <video>s thrown into website backgrounds, I can see just eating the extra bandwidth costs and keeping it all h264 as a valid strategy.
    - Dylan16807 2mo ago
      Well a lot of the improvements cost more computation. I wouldn't be surprised if low-computation video had only improved slightly, but it seems like it has improved a lot. The advantage of h264 is hardware adoption much more than actually being computationally cheap, it seems? Well, maybe. The thing is, I don't know how the decoders scale. Can you make a decoder for a constrained AV1/h.265 in the same hardware space or maximum CPU budget as an h.264 decoder and still see very big advantages?
    - duskwuff 2mo ago
      Once the patents expire, I expect to see H.264 become the JPEG of video codecs - it may not be the most efficient format available, but it's widely supported, relatively easy to encode/decode, and good enough quality for many use cases. (MP3 is in a similar place. AAC and Opus are technically superior, but everything supports MP3.)
- userbinator 2mo ago
  just that I think the h264 long tail will be one of the longest long tails of a video codec. I think MPEG-1 and 2 (from which MP3 comes) might be even longer.
  - drmpeg 2mo ago
    MPEG-2 video is still very much alive. In the US, almost all terrestrial OTA broadcast (ATSC 1.0) and all SD channels on cable systems. Even Comcast VOD was all MPEG-2 (including HD) until they shut down the QAM version in 2023.
    - winstonwinston 2mo ago
      In Europe too, for Sd via DVB-C/T2. It is however non-existent for OTT.
    - jchw 2mo ago
      The way I see it, if you were to draw two graph lines showing the prevalence of H.262 and H.264 over time, it would show of course that H.262 has lasted much longer so far just by virtue of it being older yet still in active use, but the area under the H.264 line is probably already or tracking on being much larger, depending on many factors, such as how you quantify "prevalence". I don't see H.262 as becoming completely irrelevant any time soon, so it's clearly had a hell of a run, but I expect H.264 to wind up having a similarly long tail. I am not sure I'd bet on it being longer, but I wouldn't bet against it either. I'm only of the opinion that H.264 will have one of the longest of any video codec, but perhaps not the longest.
      - drmpeg 2mo ago
        For sure. Just YouTube alone is a vast ocean of H.264 bits.
userbinator 2mo ago
DivX/XviD yes, but not H.263. Patents for H.263 expired long ago. This is MPEG-4 ASP, which is a superset of H.263. (Source: have written a H.263 decoder as a learning exercise.)
- anthk 2mo ago
  H.263 was used as the former codec of Netmeeting (and maybe some other VoIP software too).
  - userbinator 2mo ago
    H.263 was definitely a standard in videoconferencing and low-bitrate video in general (YouTube and many other streaming sutes used FLV1, which was a variant of H.263) prior to H.264.

midnightbobarun 2mo ago

Very nice, I hope this means increased support for it in open source projects

theandrewbailey 2mo ago

> While the US and EU patents around MPEG-4 Part 2 expired in recent years, until today there remained one patent still active in Brazil. Since MPEG-this Part that is hard to keep straight, this is the MPEG standard backing Xvid and DivX codecs. So let's encode video and torrent like it's 2002!

CharlesW 2mo ago
FWIW, Via Licensing listed this patent as standards-essential to several MPEG codecs (AVC/H.264, HEVC/H.265, VVC/H.266), and it's also listed for VC-1 (a SMPTE standard).
- p1mrx 2mo ago
  The openness of those codecs depend on the last patent expiring, not the first.
  - CharlesW 2mo ago
    Of course.
dylan604 2mo ago
> So let's encode video and torrent like it's 2002! Let's have a meet up a Circuit City!!
- xattt 2mo ago
  I’m not sure how to get there. Do you have Microsoft Streets and Trips directions I can print off?
ColdStream 2mo ago
Time do get out the old CD of The.Matrix.1999-DVDrip(aXXo).Xvid.
- anal_reactor 2mo ago

rcarmo 2mo ago

Good. I have been poking at rcarmo/go-264 but was hesitant to tackle encoding :)

rurban 2mo ago

But they sold xvid decoders already for the last 20 years. Every Chinese dvd players could play xvid already. Just not >1 b-frames

at1as 2mo ago

Moving in the right direction, but unfortunately it's still quite a few years before H264 is off patent, globally: https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_MPEG-4_AVC_expired_yet%3F https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M... At which point, the continued march towards higher resolution and bandwidth video will probably limit their utility

ronsor 2mo ago
> At which point, the continued march towards higher resolution and bandwidth video will probably limit their utility Just as the patent system intended: maximum extraction was enabled while the product was useful.
- goalieca 2mo ago
  Now if only copyright didn't last forever..
  - ronsor 2mo ago
    The Berne Convention has permanently burned us there
    - account42 2mo ago
      Past international agreements are only as permanent as the people alive now agree they are. Yes this one will be very hard to change but that's just because there are many powerful interests that want to keep it this way.
  - snark_attack 2mo ago
    [dead]
  - cornell532 2mo ago
    Why? The Beatles really wrote and performed Strawberry Fields. Why are you entitled to it for gratis? It’s a taking for public benefit. Ancient writings like Plato or Marcus Aurelius somehow feel different. But high-fidelity live recording feels like it shouldn’t go into the public domain. It’s someone’s commercial work product

anthk 2mo ago

Too late; software patents in Europe were void so most people grabbed XVID/DIVX encoders from Debian Multimedia. PKF and the like. And, yes, I remember w32codecs until MPlayer's bundled set superseded them. And mencoder for sure ;)

nikanj 2mo ago

It's so weird to look at the patents list for H.264 and see multiple patents granted after the H.264 specification was released. Was the situation similar for MPEG-4?

userbinator 2mo ago
Those patents may not necessarily be required to encode or decode, if they're just a new way to implement some part of the specification.
wmf 2mo ago
The patent pool system is really bad. There's no incentive to weed out bad patents.

1317 2mo ago

as everyone is saying, this is Xvid, NOT h264. pretty poor reporting by phoronix here https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_MPEG-4_AVC_expired_yet%3F https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M...

aboardRat4 2mo ago

When MPEG-5?

LtdJorge 2mo ago
Never, I hope
ksec 2mo ago
There is. MPEG-5 EVC Basic Profile that is Patent unencumbered, with only dozens patents that is soon to be expired, and EVC High Profile that competes with VVC in terms of quality. There is also LCEVC ( Not connected with EVC ) that is a scaling layer on top of existing video codec. You can apply LCEVC on top of EVC, VVC, HEVC or AVC as well as AV1 and AV2. LCEVC has been selected by Brail as part of their TV 3.0 technology stack on top VVC. While there are momentum on LCEVC, I actually wished EVC gained more momentum. I guess everything is too late now.

hulitu 2mo ago

That's why they push for other codecs.