15 ms·

As someone who hasn't had any exposure to the human stories behind mpeg before, it feels to me like it's been a force for evil since long before 2020. Patents o

As someone who hasn't had any exposure to the human stories behind mpeg before, it feels to me like it's been a force for evil since long before 2020. Patents on h264, h265, and even mp3 have been holding the industry back for decades. Imagine what we might have if their iron grip on codecs was broken.

jbverschoor 1y ago
Enough codecs out there. Just no adoption.
- wheybags 1y ago
  Yes, because mpeg got there first, and now their dominance is baked into silicon with hardware acceleration. It's starting to change at last but we have a long way to go. That way would be a lot easier if their patent portfolio just died.
- egeozcan 1y ago
  This might be an oversimplification, but as a consumer, I think I see a catch-22 for new codecs. Companies need a big incentive to invest in them, which means the codec has to be technically superior and safe from hidden patent claims. But the only way to know if it's safe is for it to be widely used for a long time. Of course, it can't get widely used without company support in the first place. So, while everyone waits, the technology is no longer superior, and the whole thing fizzles out.
  - Taek 1y ago
    Companies only need a big incentive to invest in new codecs because creating a codec that has a simple incremental improvement would violate existing patents.
  - jbverschoor 1y ago
    Jxl has been around for years. Av1 for 7 The problem is every platform wants to force their own codec, and get earn royalties from the rest of the world. They literally sabotaging it. Jxl support even got removed from chrome. Investment in adopting in software is next to 0. In hardware it’s a different story, and I’m not sure to what extent which codec can be properly accelerated
- TiredOfLife 1y ago
  Because every codec has 3+ different patent pools wanting rent. Each with different terms.
- rs186 1y ago
  Not all codecs are equal, and to be honest, most are probably not optimized/suitable for today's applications, otherwise Google wouldn't have invented their own codec (which then gets adopted widely, fortunately).
mike_hearn 1y ago
Possibly, nothing. Codec development is slow and expensive. Free codecs only came along at all because Google decided to subsidize development but that became possible only 15 years or so after MPEG was born, and it's hardly a robust strategy. Plus free codecs were often built by acquiring companies that had previously been using IP licensing as a business model rather than from-scratch development.
- wheybags 1y ago
  It's not just about new codecs. There's also people making products that would use codecs just deciding not to because of the patent hassle.
- newsclues 1y ago
  This is the sort of project that should be developed and released via open source from academia. Audio and video codecs, document formats like PDF, are all foundational to computing and modern life from government to business, so there is a great incentive to make it all open, and free.
  - oblio 1y ago
    You're also describing technologies with universal use and potential for long term rent seeking. Basically MBA drool material.
    - newsclues 1y ago
      Yeah, and if MBAs want to reap that reward, they need to fund the development exclusively without government funding.
  - mike_hearn 1y ago
    Universities love patent licensing. I don't think academia is the solution you're looking for.
    - yxhuvud 1y ago
thinkingQueen 1y ago
Who would develop those codecs? A good video coding engineer costs about 100-300k USD a year. The really good ones even more. You need a lot of them. JVET has an attendance of about 350 such engineers each meeting (four times a year). Not to mention the computer clusters to run all the coding sims, thousands and thousands of CPUs are needed per research team. People who are outside the video coding industry do not understand that it is an industry. It’s run by big companies with large R&D budgets. It’s like saying ”where would we be with AI if Google, OpenAI and Nvidia didn’t have an iron grip”. MPEG and especially JVET are doing just fine. The same companies and engineers who worked on AVC, HEVC and VVC are still there with many new ones especially from Asia. MPEG was reorganized because this Leonardo guy became an obstacle, and he’s been angry about ever since. Other than that I’d say business as usual in the video coding realm.
- roenxi 1y ago
  > It’s like saying ”where would we be with AI if Google, OpenAI and Nvidia didn’t have an iron grip”. We'd be where we are. All the codec-equivalent aspects of their work are unencumbered by patents and there are very high quality free models available in the market that are just given away. If the multimedia world had followed the Google example it'd be quite hard to complain about the codecs.
  - thinkingQueen 1y ago
    That’s hardly true. Nvidia’s tech is covered by patents and licenses. Why else would it be worth 4.5 trillion dollars? The top AI companies use very restrictive licenses. I think it’s actually the other way around and AI industry will actually end up following the video coding industry when it comes to patents, royalties, licenses etc.
    - oblio 1y ago
      I imagine a chunk of it is also covered by trade secrets and NDAs.
    - roenxi 1y ago
fidotron 1y ago
The fact h264 and h265 are known by those terms is key to the other part of the equation: the ITU Video Coding Experts Group has become the dominant forum for setting standards going back to at least 2005.
Reason077 1y ago
> "Patents on h264, h265, and even mp3 have been holding the industry back for decades. Imagine what we might have if their iron grip on codecs was broken." Has AV1 solved this, to some extent? Although there are patent claims against it (patents for technologies that are fundamental to all the modern video codecs), it still seems better than the patent & licensing situation for h264 / h265.
- afroboy 1y ago
  The power of H264 and H265 comes from pirates, and since AV1 team don't work with pirates then it will always be inferior to H265. Just check pirated releases of TV shows and movies.
philistine 1y ago
At least for MP3, our collective nightmare is over. MP3 is completely patent-unencumbered and can be used freely.
doublerabbit 1y ago
To me, 2007 is when the evil forces really took hold. mySpace era was the last fun era. Everything after that kind of lacks.
riedel 1y ago
MPEG-7 includes a binary XML standard [0] which is quite useful IMHO in comparison to others (I think it is used in DVB Meta data streams). But beyond patents it is even hard to find open documentation of BIM. I think the group was technically quite competent in comparison with other standard groups, but the business models around it really turn me off. [0] https://mpeg.chiariglione.org/standards/mpeg-7/reference-software.html https://mpeg.chiariglione.org/standards/mpeg-7/reference-sof... EDIT: Here is the Wikipedia page of BiM which evidently made it even into an ISO Standard [1] [1] https://en.m.wikipedia.org/wiki/BiM https://en.m.wikipedia.org/wiki/BiM
- bokchoi 1y ago
  Interesting. I've used EXI in a past project but I hadn't heard of BiM.
  - riedel 1y ago
    More than 10 years ago. I was looking into using binary XML on really ressource constraint embedded devices (actually I started to dive into the whole rabbit hole of grammar based compression on hedges in my PhD). I tried to like EXI, but realized that it wasn't really designed for embedded applications. I ended up implementing binary representations very much inspired by BiM.
fweimer 1y ago
The really silly part is that even if you have a license from MPEG LA for your product, you still have to put in a notice like this: THIS PRODUCT IS LICENSED UNDER THE AVC PATENT PORTFOLIO LICENSE FOR THE PERSONAL AND NON-COMMERCIAL USE OF A CONSUMER TO (I) ENCODE VIDEO IN COMPLIANCE WITH THE AVC STANDARD ("AVC VIDEO") AND/OR (II) DECODE AVC VIDEO THAT WAS ENCODED BY A CONSUMER ENGAGED IN A PERSONAL AND NON-COMMERCIAL ACTIVITY AND/OR WAS OBTAINED FROM A VIDEO PROVIDER LICENSED TO PROVIDE AVC VIDEO. NO LICENSE IS GRANTED OR SHALL BE IMPLIED FOR ANY OTHER USE. ADDITIONAL INFORMATION MAY BE OBTAINED FROM MPEG LA, L.L.C. SEE HTTP://WWW.MPEGLA.COM It's unclear whether this license covers videoconferencing for work purposes (where you are paid, but not specifically to be on that call). It seems to rule out remote tutoring. MPEG LA probably did not have much choice here because this language requirement (or language close to it) for outgoing patent licenses is likely part of their incoming patent license agreements. It's probably impossible at this point to renegotiate and align the terms with how people actually use video codecs commercially today. But it means that you can't get a pool license from MPEG LA that covers commercial videoconferencing, you'd have to negotiate separately with the individual patent holders.
wmf 1y ago
Yeah, he ran an incubator for patent trolls for 30 years and now the patent trolls have eaten his face.

jbverschoor 1y ago

Enough codecs out there. Just no adoption.

wheybags 1y ago
Yes, because mpeg got there first, and now their dominance is baked into silicon with hardware acceleration. It's starting to change at last but we have a long way to go. That way would be a lot easier if their patent portfolio just died.
egeozcan 1y ago
This might be an oversimplification, but as a consumer, I think I see a catch-22 for new codecs. Companies need a big incentive to invest in them, which means the codec has to be technically superior and safe from hidden patent claims. But the only way to know if it's safe is for it to be widely used for a long time. Of course, it can't get widely used without company support in the first place. So, while everyone waits, the technology is no longer superior, and the whole thing fizzles out.
- Taek 1y ago
  Companies only need a big incentive to invest in new codecs because creating a codec that has a simple incremental improvement would violate existing patents.
- jbverschoor 1y ago
  Jxl has been around for years. Av1 for 7 The problem is every platform wants to force their own codec, and get earn royalties from the rest of the world. They literally sabotaging it. Jxl support even got removed from chrome. Investment in adopting in software is next to 0. In hardware it’s a different story, and I’m not sure to what extent which codec can be properly accelerated
TiredOfLife 1y ago
Because every codec has 3+ different patent pools wanting rent. Each with different terms.
rs186 1y ago
Not all codecs are equal, and to be honest, most are probably not optimized/suitable for today's applications, otherwise Google wouldn't have invented their own codec (which then gets adopted widely, fortunately).

mike_hearn 1y ago

Possibly, nothing. Codec development is slow and expensive. Free codecs only came along at all because Google decided to subsidize development but that became possible only 15 years or so after MPEG was born, and it's hardly a robust strategy. Plus free codecs were often built by acquiring companies that had previously been using IP licensing as a business model rather than from-scratch development.

wheybags 1y ago
It's not just about new codecs. There's also people making products that would use codecs just deciding not to because of the patent hassle.
newsclues 1y ago
This is the sort of project that should be developed and released via open source from academia. Audio and video codecs, document formats like PDF, are all foundational to computing and modern life from government to business, so there is a great incentive to make it all open, and free.
- oblio 1y ago
  You're also describing technologies with universal use and potential for long term rent seeking. Basically MBA drool material.
  - newsclues 1y ago
    Yeah, and if MBAs want to reap that reward, they need to fund the development exclusively without government funding.
- mike_hearn 1y ago
  Universities love patent licensing. I don't think academia is the solution you're looking for.
  - yxhuvud 1y ago

thinkingQueen 1y ago

Who would develop those codecs? A good video coding engineer costs about 100-300k USD a year. The really good ones even more. You need a lot of them. JVET has an attendance of about 350 such engineers each meeting (four times a year). Not to mention the computer clusters to run all the coding sims, thousands and thousands of CPUs are needed per research team. People who are outside the video coding industry do not understand that it is an industry. It’s run by big companies with large R&D budgets. It’s like saying ”where would we be with AI if Google, OpenAI and Nvidia didn’t have an iron grip”. MPEG and especially JVET are doing just fine. The same companies and engineers who worked on AVC, HEVC and VVC are still there with many new ones especially from Asia. MPEG was reorganized because this Leonardo guy became an obstacle, and he’s been angry about ever since. Other than that I’d say business as usual in the video coding realm.

roenxi 1y ago
> It’s like saying ”where would we be with AI if Google, OpenAI and Nvidia didn’t have an iron grip”. We'd be where we are. All the codec-equivalent aspects of their work are unencumbered by patents and there are very high quality free models available in the market that are just given away. If the multimedia world had followed the Google example it'd be quite hard to complain about the codecs.
- thinkingQueen 1y ago
  That’s hardly true. Nvidia’s tech is covered by patents and licenses. Why else would it be worth 4.5 trillion dollars? The top AI companies use very restrictive licenses. I think it’s actually the other way around and AI industry will actually end up following the video coding industry when it comes to patents, royalties, licenses etc.
  - oblio 1y ago
    I imagine a chunk of it is also covered by trade secrets and NDAs.
  - roenxi 1y ago

fidotron 1y ago

The fact h264 and h265 are known by those terms is key to the other part of the equation: the ITU Video Coding Experts Group has become the dominant forum for setting standards going back to at least 2005.

Reason077 1y ago

> "Patents on h264, h265, and even mp3 have been holding the industry back for decades. Imagine what we might have if their iron grip on codecs was broken." Has AV1 solved this, to some extent? Although there are patent claims against it (patents for technologies that are fundamental to all the modern video codecs), it still seems better than the patent & licensing situation for h264 / h265.

afroboy 1y ago
The power of H264 and H265 comes from pirates, and since AV1 team don't work with pirates then it will always be inferior to H265. Just check pirated releases of TV shows and movies.

philistine 1y ago

At least for MP3, our collective nightmare is over. MP3 is completely patent-unencumbered and can be used freely.

doublerabbit 1y ago

To me, 2007 is when the evil forces really took hold. mySpace era was the last fun era. Everything after that kind of lacks.

riedel 1y ago

MPEG-7 includes a binary XML standard [0] which is quite useful IMHO in comparison to others (I think it is used in DVB Meta data streams). But beyond patents it is even hard to find open documentation of BIM. I think the group was technically quite competent in comparison with other standard groups, but the business models around it really turn me off. [0] https://mpeg.chiariglione.org/standards/mpeg-7/reference-software.html https://mpeg.chiariglione.org/standards/mpeg-7/reference-sof... EDIT: Here is the Wikipedia page of BiM which evidently made it even into an ISO Standard [1] [1] https://en.m.wikipedia.org/wiki/BiM https://en.m.wikipedia.org/wiki/BiM

bokchoi 1y ago
Interesting. I've used EXI in a past project but I hadn't heard of BiM.
- riedel 1y ago
  More than 10 years ago. I was looking into using binary XML on really ressource constraint embedded devices (actually I started to dive into the whole rabbit hole of grammar based compression on hedges in my PhD). I tried to like EXI, but realized that it wasn't really designed for embedded applications. I ended up implementing binary representations very much inspired by BiM.

fweimer 1y ago

The really silly part is that even if you have a license from MPEG LA for your product, you still have to put in a notice like this: THIS PRODUCT IS LICENSED UNDER THE AVC PATENT PORTFOLIO LICENSE FOR THE PERSONAL AND NON-COMMERCIAL USE OF A CONSUMER TO (I) ENCODE VIDEO IN COMPLIANCE WITH THE AVC STANDARD ("AVC VIDEO") AND/OR (II) DECODE AVC VIDEO THAT WAS ENCODED BY A CONSUMER ENGAGED IN A PERSONAL AND NON-COMMERCIAL ACTIVITY AND/OR WAS OBTAINED FROM A VIDEO PROVIDER LICENSED TO PROVIDE AVC VIDEO. NO LICENSE IS GRANTED OR SHALL BE IMPLIED FOR ANY OTHER USE. ADDITIONAL INFORMATION MAY BE OBTAINED FROM MPEG LA, L.L.C. SEE HTTP://WWW.MPEGLA.COM It's unclear whether this license covers videoconferencing for work purposes (where you are paid, but not specifically to be on that call). It seems to rule out remote tutoring. MPEG LA probably did not have much choice here because this language requirement (or language close to it) for outgoing patent licenses is likely part of their incoming patent license agreements. It's probably impossible at this point to renegotiate and align the terms with how people actually use video codecs commercially today. But it means that you can't get a pool license from MPEG LA that covers commercial videoconferencing, you'd have to negotiate separately with the individual patent holders.

wmf 1y ago

Yeah, he ran an incubator for patent trolls for 30 years and now the patent trolls have eaten his face.