11 ms·
Leonardo Chiariglione – Co-founder of MPEG
- wheybags 1y agoAs someone who hasn't had any exposure to the human stories behind mpeg before, it feels to me like it's been a force for evil since long before 2020. Patents on h264, h265, and even mp3 have been holding the industry back for decades. Imagine what we might have if their iron grip on codecs was broken.
- jbverschoor 1y agoEnough codecs out there. Just no adoption.
- wheybags 1y agoYes, because mpeg got there first, and now their dominance is baked into silicon with hardware acceleration. It's starting to change at last but we have a long way to go. That way would be a lot easier if their patent portfolio just died.
- egeozcan 1y agoThis might be an oversimplification, but as a consumer, I think I see a catch-22 for new codecs. Companies need a big incentive to invest in them, which means the codec has to be technically superior and safe from hidden patent claims. But the only way to know if it's safe is for it to be widely used for a long time. Of course, it can't get widely used without company support in the first place. So, while everyone waits, the technology is no longer superior, and the whole thing fizzles out.
- Taek 1y agoCompanies only need a big incentive to invest in new codecs because creating a codec that has a simple incremental improvement would violate existing patents.
- jbverschoor 1y agoJxl has been around for years. Av1 for 7 The problem is every platform wants to force their own codec, and get earn royalties from the rest of the world. They literally sabotaging it. Jxl support even got removed from chrome. Investment in adopting in software is next to 0. In hardware it’s a different story, and I’m not sure to what extent which codec can be properly accelerated
- TiredOfLife 1y agoBecause every codec has 3+ different patent pools wanting rent. Each with different terms.
- rs186 1y agoNot all codecs are equal, and to be honest, most are probably not optimized/suitable for today's applications, otherwise Google wouldn't have invented their own codec (which then gets adopted widely, fortunately).
- mike_hearn 1y agoPossibly, nothing. Codec development is slow and expensive. Free codecs only came along at all because Google decided to subsidize development but that became possible only 15 years or so after MPEG was born, and it's hardly a robust strategy. Plus free codecs were often built by acquiring companies that had previously been using IP licensing as a business model rather than from-scratch development.
- wheybags 1y agoIt's not just about new codecs. There's also people making products that would use codecs just deciding not to because of the patent hassle.
- newsclues 1y agoThis is the sort of project that should be developed and released via open source from academia. Audio and video codecs, document formats like PDF, are all foundational to computing and modern life from government to business, so there is a great incentive to make it all open, and free.
- oblio 1y agoYou're also describing technologies with universal use and potential for long term rent seeking. Basically MBA drool material.
- newsclues 1y agoYeah, and if MBAs want to reap that reward, they need to fund the development exclusively without government funding.
- mike_hearn 1y agoUniversities love patent licensing. I don't think academia is the solution you're looking for.
- yxhuvud 1y ago
- thinkingQueen 1y agoWho would develop those codecs? A good video coding engineer costs about 100-300k USD a year. The really good ones even more. You need a lot of them. JVET has an attendance of about 350 such engineers each meeting (four times a year). Not to mention the computer clusters to run all the coding sims, thousands and thousands of CPUs are needed per research team. People who are outside the video coding industry do not understand that it is an industry. It’s run by big companies with large R&D budgets. It’s like saying ”where would we be with AI if Google, OpenAI and Nvidia didn’t have an iron grip”. MPEG and especially JVET are doing just fine. The same companies and engineers who worked on AVC, HEVC and VVC are still there with many new ones especially from Asia. MPEG was reorganized because this Leonardo guy became an obstacle, and he’s been angry about ever since. Other than that I’d say business as usual in the video coding realm.
- roenxi 1y ago> It’s like saying ”where would we be with AI if Google, OpenAI and Nvidia didn’t have an iron grip”. We'd be where we are. All the codec-equivalent aspects of their work are unencumbered by patents and there are very high quality free models available in the market that are just given away. If the multimedia world had followed the Google example it'd be quite hard to complain about the codecs.
- thinkingQueen 1y agoThat’s hardly true. Nvidia’s tech is covered by patents and licenses. Why else would it be worth 4.5 trillion dollars? The top AI companies use very restrictive licenses. I think it’s actually the other way around and AI industry will actually end up following the video coding industry when it comes to patents, royalties, licenses etc.
- fidotron 1y agoThe fact h264 and h265 are known by those terms is key to the other part of the equation: the ITU Video Coding Experts Group has become the dominant forum for setting standards going back to at least 2005.
- Reason077 1y ago> "Patents on h264, h265, and even mp3 have been holding the industry back for decades. Imagine what we might have if their iron grip on codecs was broken." Has AV1 solved this, to some extent? Although there are patent claims against it (patents for technologies that are fundamental to all the modern video codecs), it still seems better than the patent & licensing situation for h264 / h265.
- afroboy 1y agoThe power of H264 and H265 comes from pirates, and since AV1 team don't work with pirates then it will always be inferior to H265. Just check pirated releases of TV shows and movies.
- philistine 1y agoAt least for MP3, our collective nightmare is over. MP3 is completely patent-unencumbered and can be used freely.
- doublerabbit 1y agoTo me, 2007 is when the evil forces really took hold. mySpace era was the last fun era. Everything after that kind of lacks.
- riedel 1y agoMPEG-7 includes a binary XML standard [0] which is quite useful IMHO in comparison to others (I think it is used in DVB Meta data streams). But beyond patents it is even hard to find open documentation of BIM. I think the group was technically quite competent in comparison with other standard groups, but the business models around it really turn me off. [0] https://mpeg.chiariglione.org/standards/mpeg-7/reference-software.html https://mpeg.chiariglione.org/standards/mpeg-7/reference-sof... EDIT: Here is the Wikipedia page of BiM which evidently made it even into an ISO Standard [1] [1] https://en.m.wikipedia.org/wiki/BiM https://en.m.wikipedia.org/wiki/BiM
- bokchoi 1y agoInteresting. I've used EXI in a past project but I hadn't heard of BiM.
- riedel 1y agoMore than 10 years ago. I was looking into using binary XML on really ressource constraint embedded devices (actually I started to dive into the whole rabbit hole of grammar based compression on hedges in my PhD). I tried to like EXI, but realized that it wasn't really designed for embedded applications. I ended up implementing binary representations very much inspired by BiM.
- fweimer 1y agoThe really silly part is that even if you have a license from MPEG LA for your product, you still have to put in a notice like this: THIS PRODUCT IS LICENSED UNDER THE AVC PATENT PORTFOLIO LICENSE FOR THE PERSONAL AND NON-COMMERCIAL USE OF A CONSUMER TO (I) ENCODE VIDEO IN COMPLIANCE WITH THE AVC STANDARD ("AVC VIDEO") AND/OR (II) DECODE AVC VIDEO THAT WAS ENCODED BY A CONSUMER ENGAGED IN A PERSONAL AND NON-COMMERCIAL ACTIVITY AND/OR WAS OBTAINED FROM A VIDEO PROVIDER LICENSED TO PROVIDE AVC VIDEO. NO LICENSE IS GRANTED OR SHALL BE IMPLIED FOR ANY OTHER USE. ADDITIONAL INFORMATION MAY BE OBTAINED FROM MPEG LA, L.L.C. SEE HTTP://WWW.MPEGLA.COM It's unclear whether this license covers videoconferencing for work purposes (where you are paid, but not specifically to be on that call). It seems to rule out remote tutoring. MPEG LA probably did not have much choice here because this language requirement (or language close to it) for outgoing patent licenses is likely part of their incoming patent license agreements. It's probably impossible at this point to renegotiate and align the terms with how people actually use video codecs commercially today. But it means that you can't get a pool license from MPEG LA that covers commercial videoconferencing, you'd have to negotiate separately with the individual patent holders.
- wmf 1y agoYeah, he ran an incubator for patent trolls for 30 years and now the patent trolls have eaten his face.
- dostick 1y agoThe article does not give much beyond what you already read in the title. What obscure forces and how? Isn’t it an open standards non-profit organisation, then what could possible hinder it? Maybe because technologically closed standards became better and nonprofit project has no resources to compete with commercial standards? USB Alliance have been able to work things out, so maybe compression standards should be developed in similar way?
- baobun 1y agoSupposedly the whole story is told in their linked book.
- eggspurt 1y agoFrom Leonardo, who founded MPEG, on the page linked: "Even before it has ceased to exists, the MPEG engine had run out of steam – technology- and business wise. The same obscure forces that have hijacked MPEG had kept it hostage to their interests impeding its technical development and keeping it locked to outmoded Intellectual Property licensing models delaying market adoption of MPEG standards. Industry has been strangled and consumers have been deprived of the benefits of new technologies. From facilitators of new opportunities and experiences, MPEG standards have morphed from into roadblocks."
- dostick 1y agoExactly. That passage only making it more confusing.
- karel-3d 1y agoI... don't understand how AI related to video codecs. Maybe because I don't understand either video codecs or AI on a deeper level.
- bjoli 1y agoIt is like upscaling. If you could train AI to "upscale" your audio or video you could get away with sending a lot less data. It is already being done with quite amazing results for audio.
- jl6 1y agoIt has long been recognised that the state of the art in data compression has much in common with the state of the art in AI, for example: http://prize.hutter1.net/ http://prize.hutter1.net/ https://bellard.org/nncp/ https://bellard.org/nncp/
- ddtaylor 1y agoSome view these as so interconnected that they will say LLMs are "just" compression.
- pjc50 1y agoWhich is an interesting view when applied to the IP. I think it's relatively uncontroversial that an MP4 file which "predicts" a Disney movie which it was "trained on" is a derived work. Suppose you have an LLM which was trained on a fairly small set of movies and you could produce any one on demand; would that be treated as a derived work? If you have a predictor/compressor LLM which was trained on all the movies in the world, would that not also be infringement?
- mr_toad 1y agoMP4s are compressed data, not a compression algorithm. An MP4 (or any compressed data) is not a “prediction”, it is the difference between what was predicted and what you’re trying to compress. An LLM is (or can be used) as a compression algorithm, but it is not compressed data. It is possible to have an overfit algorithm exactly predict (or reproduce) an output, but it’s not possible for one to reproduce all the outputs due to the pigeonhole principle. To reiterate - LLMs are not compressed data.
- ZeroGravitas 1y agoThere's nothing obscure about them. His comment immediately after describes exactly what happened: > Even before it has ceased to exists, the MPEG engine had run out of steam – technology- and business wise. The same obscure forces that have hijacked MPEG had kept it hostage to their interests impeding its technical development and keeping it locked to outmoded Intellectual Property licensing models delaying market adoption of MPEG standards. Industry has been strangled and consumers have been deprived of the benefits of new technologies. From facilitators of new opportunities and experiences, MPEG standards have morphed from into roadblocks. Big companies abused the setup that he was responsible for. Gentlemen's agreements to work together for the benefit of all got gamed into patent landmines and it happened under his watch. Even many of the big corps involved called out the bullshit, notably Steve Jobs refusing to release a new Quicktime till they fixed some of the most egregious parts of AAC licencing way back in 2002. https://www.zdnet.com/article/apple-shuns-mpeg-4-licensing-terms/ https://www.zdnet.com/article/apple-shuns-mpeg-4-licensing-t...
- scotty79 1y ago> The same obscure forces that have hijacked MPEG had kept it hostage to their interests impeding its technical development and keeping it locked to outmoded Intellectual Property licensing models delaying market adoption of MPEG standards. Industry has been strangled and consumers have been deprived of the benefits of new technologies. Copyright is cancer. The faster AI industry is going to run it into the ground, the better.
- knome 1y agoThis has nothing to do with copyright. It is an issue of patents.
- rurban 1y agoDoes he talk about Fraunhofer there? The guys, subsidized by German taxpayers, starting to charge license or patent fees. Or is it MPEG LA? https://wiki.endsoftwarepatents.org/wiki/MPEG_LA https://wiki.endsoftwarepatents.org/wiki/MPEG_LA
- ronsor 1y agoI hate copyright too, but this is about patents. Software patents are also cancer.
- scotty79 1y agoI think if IP rights holders were mandated to pay property tax it would make the system much healthier.
- londons_explore 1y agoThis. You should have to declare the value of a patent, and pay 1% of that value every year to the government. Anyone else can force-purchase it for that value, but leaving you with a free perpetual license.
- LeafItAlone 1y agoWouldn’t that only help the “big guys” who can afford to pay the tax?
- MyOutfitIsVague 1y agoPresumably the tax would be based on some estimated value of the property, and affordability would therefore scale.
- marcodiego 1y ago> My Christian Catholic education made and still makes me think that everybody should have a mission that extends beyond their personal interests. I remember this same guy complaining investments in the MPEG extortionist group would disappear because they couldn't fight against AV1. He was part of a patent Mafia is is only lamenting he lost power. Hypocrisy in its finest form.
- maxloh 1y agoAny link to his comment?
- marcodiego 1y ago> all the investments (collectively hundreds of millions USD) made by the industry for the new video codec will go up in smoke and AOM’s royalty free model will spread to other business segments as well. https://blog.chiariglione.org/a-crisis-the-causes-and-a-solution/ https://blog.chiariglione.org/a-crisis-the-causes-and-a-solu... He is not a coder, not a researcher, he is only part of the worst game there is in this industry: a money maker from patents and "standards" you need to pay for to use, implement or claim compatibility.
- DragonStrength 1y agoYou missed the first part of that quote: > At long last everybody realises that the old MPEG business model is now broke And the entire post is about how dysfunctional MPEG is and how AOM rose to deal with it. It is tragic to waste so much time and money only to produce nothing. He's criticizing the MPEG group and their infighting. He's literally criticizing MPEG's licensing model and the leadership of the companies in MPEG. He's an MPEG member saying MPEG's business model is broken yet no one has a desire to fix it, so it will be beaten by a competitor. Would you not want to see your own organization reform rather than die? Reminder AOM is a bunch of megacorps with profit motive too, which is why he thinks this ultimately leads to stalled innovation: > My concerns are at a different level and have to do with the way industry at large will be able to access innovation. AOM will certainly give much needed stability to the video codec market but this will come at the cost of reduced if not entirely halted technical progress. There will simply be no incentive for companies to develop new video compression technologies, at very significant cost because of the sophistication of the field, knowing that their assets will be thankfully – and nothing more – accepted and used by AOM in their video codecs. > Companies will slash their video compression technology investments, thousands of jobs will go and millions of USD of funding to universities will be cut. A successful “access technology at no cost” model will spread to other fields. Money is the motivator. Figuring out how to reward investment in pushing the technology forward is his concern. It sounds like he is open to suggestions.
- selvan 1y agoMay be, we are couple of years away from experiencing patent free video codecs based on deep learning. DCVC-RT (https://github.com/microsoft/DCVC https://github.com/microsoft/DCVC) - A deep learning based video codec claims to deliver 21% more compression than h266. One of the compelling edge AI usecases is to create deep learning based audio/video codecs on consumer hardwares. One of the large/enterprise AI usecases is to create a coding model that generates deep learning based audio/video codecs for consumer hardwares.
- _bent 1y agohttps://mpai.community/standards/mpai-spg https://mpai.community/standards/mpai-spg This makes zero sense, right? Even if this was applicable, why would it need a standard? There is no interoperability between game servers of different games
- kmeisthax 1y ago[dead]
- mananaysiempre 1y agoOne detail for context: when “closing” MPEG, he also deleted all of its all pages and materials and redirected them to the AI stuff.
- gcr 1y agoGoodbye MPEG group, and to be frank, good riddance I think. I'm glad that open codecs are now taking over on the frontier of SOTA encoding. Maybe these sorts of handshake agreements and industry collaboration were necessary to get things rolling in 198x. If so, then I thank the MPEG group for starting that work. But by 2005 or so when DivX and XviD and h264 were heating up, it was time to move beyond that model towards open interoperability.
- zazazx 1y ago“…and industry to exploit.” And, boy howdy, they did.
- kouru225 1y agoSo what’s the take on his new organization MPAI? I don’t know much about writing codecs… would love to hear someone’s take on the organization.
- Merrill 1y agoMPEG's patent policy was not its own. The Moving Picture Experts Group was Working Group 11 of Sub Committee 2 of ISO-IEC Joint Technical Committee 1. Other Working Groups in SC2 were the Joint Photographic Experts Group, the Joint Bi-Level Image Group, and the Multimedia/Hypermedia Experts Group. Therefore, the patent policy was the ISO-IEC policy for including patented technology in their standards. MPEG was also joint with the video conferencing standards group within the CCITT (now International Telecommunications Union), which generally required FRAND declarations from patent holders. My recollection is that MPEG-LA was set up as a clearing house so that implementers could go to one licensing organization, rather than negotiating with each patent owner individually. All the patents for MPEG 1 and MPEG 2 must be expired by now. Besides patent gridlock, there is a fundamental economic problem with developing new video coding algorithms. It's very difficult to develop an algorithm that will halve the bit rate for the same quality, to get it implemented in hardware products an software, and to introduce it broadly in the existing video services infrastructure. Plus, doubling the compression is likely to more than double the processing required. On the other hand, within a couple of years the network engineers will double the bit rate for the same cost, and the storage engineers will double the storage for the same cost. They, like processing, follow their own Moore's Law. So reducing the cost by improving codecs is more expensive and takes more effort and time than just waiting for the processor, storage and networking cost reductions. At least that's been true over the 3 decades since MPEG 2.