7 ms·

A JavaScript H.264 decoder

142 points by tnajdek 15y ago

hmottestad 15y ago
Running on a macbook pro with i5 dual core 2.4 ghz, it uses 1 core at 100% in firefox and manages: 1.7 fps It's cool, but I don't see any real world applications. Anyone got any ideas?
- ck2 15y ago
  I think it's more of an (impressive) mental exercise - if the code can be made straightforward enough to run on javascript, the future applications can only grow from there.
  - robert-boehnke 15y ago
    It's the Android encoder written in c++ cross compiled to JavaScript, not much of a mental exercise. Not to say it's not cool :)
    - ZeroGravitas 15y ago
      They have a hand-coded javascript version in progress too: https://github.com/mbebenita/Broadway/tree/master/Play https://github.com/mbebenita/Broadway/tree/master/Play
- salmanapk 15y ago
  Strange.. It was running at 4 fps on my Pentium 4 lol
  - hmottestad 15y ago
    hmmm.... What browser, OS and what Pentium 4 processor? And for how long did you run the video? Mine drops to 1.7 after a few seconds, the first few frames are dark so in the beginning I get 6 fps.
    - salmanapk 15y ago
      Nightly, XP, 2.4 Ghz. Nearly 5 Minutes.. Here it starts at like 0.6 fps, and then runs at 4-7 fps onwards.
      - hmottestad 15y ago
        Must be something with Nightly then, I'm just running the regular version. Says a lot about the importance of software over hardware :)
        shaver 15y ago
        It definitely needs nightly. I'm not sure what's up with salmanapk's setup, will need to dig into it.
- shaver 15y ago
  You need Firefox nightly for it to run well, because it depends on the hybrid static/dynamic type inference work and some other optimizations. I believe that Brendan was demoing it from a Mac laptop of some kind. As far as applications, it will let content providers who want to ship H.264 and pay the license fees do so without requiring all browser developers to pay such fees. (For Firefox it would be a meaningful portion of Mozilla's engineering budget.) And it's a pretty compelling demo, IMO, of the fact that we're not done with JS performance yet, and that people don't need to be running to Dart or NaCl or other rip-and-replace technologies in order to get great perf. There's a lot of opportunity for even better performance on the engine side as well as the library side: they started a week ago with emscripten, Closure and the libstagefright C sources...
- grayrest 15y ago
  With today's Nightly I'm getting ~20fps on a 3.4 GHz Core i7 iMac but with drops into the 7fps range on some video transitions. Surprisingly, I get roughly the same perf with a current Aurora as well but the drops only go down to ~12fps. Trying it on the released Firefox gets around 2fps.
- azakai 15y ago
  1. This works much faster on Firefox nightly (that's where we demo'd it on). Nightly has a lot of JS engine improvements that are not in stable yet (but will be in a month or two). 2. This does not use any hardware acceleration yet. This is simply compiling the Android C decoder into JS, nothing else - just a few days of work. We will now start to look at actually optimizing the code specifically for JS, and also to use GPU shaders for the relevant part of the code. Both of those can potentially make this even faster. Regarding real-world applications, the interesting possibility is for websites to ship their own codecs. As I mentioned in a comment above, imagine an anime video website that ships a video decoder optimized for that kind of content - it could be much more efficient than stock H.264. Also, if websites can ship their own decoders, they can continually improve them (unlike now where websites rely on client decoders which are not constantly being improved).
__alexs 15y ago
Apparently this can actually get decent frame rates. Although I suspect they are using the patched version of the JS interpreter mentioned on github. http://yfrog.com/nmng0z http://yfrog.com/nmng0z Still not sure how this is meant to actually be useful though. The problem with H.264 isn't availability of implementations, it's being non-free and heavily patented. I can kind of see a use for this if you are a big content provider with a bunch of cash and want to distribute video to Firefox users without transcoding but it still seems pretty derpy.
- shaver 15y ago
  Brendan's demo was running at 30fps, I believe on a Macbook Pro (I'll find out). Some content will run slower right now, which is true of all codecs AFAIK but is more so for this stuff right now. The patches linked from the github README aren't necessary to run the transpiled version that was in the demo -- it's a memory optimization that's being used in the tuned-for-JS version. As regards derpiness: it lets content distributors decide to pay for H.264, if they want, exactly, and paves the way for other codecs to be deployed by such distributors as well. It also runs the codec in a managed environment -- format decoders are often very fertile territory for exploitable bugs, since they are pretty much by definition all about pointer math and byte-poking. But the initial intent, when they decided a week ago to try it, was to push the envelope of JS performance such that we find new ways to extend said envelope.
  - pcwalton 15y ago
    It was on a MacBook Pro. We tried it on a MacBook Air too and it performed reasonably well.
    - watmough 15y ago
      My 2007 MBP manages a decent 20 fps but starts pretty slow and averages 15. That's a 2.4GHz C2D. Need to try it on my i52400.
      - mbebenita 15y ago
hackermom 15y ago
Funny hobby project.
- BrendanEich 15y ago
  ... said the dinosaur to the first mammal, re: size, fur, and all that hobby fluff.
  - hackermom 15y ago
    You just can't compare a JS library for h.264 decoding with ditto made in C. The latter is usable while the former is practically useless even for an extreme case of f.e. online, browser-based video editing. What is the point you are trying to make?
gerggerg 15y ago
Any one know what the legal implications of using a free decoder for a patented compression format are? Products like flash make it seem free as Adobe pays a flat rate to the patent holders for decoding mp3, h.264, etc. alleviating their users from having to worry about royalties. But would using this decoder, though free and open source, be infringing a patent? note: I realize it's not production code. I'm asking more about the concept.
- shaver 15y ago
  Use and distribution of implementations requires a license (though, curiously, x264 seems to get a pass), so using the decoder will require a license as with using a closed-source one.
  - DarkShikari 15y ago
    (though, curiously, x264 seems to get a pass) If x264 does, so does VLC, mplayer, ffmpeg, gstreamer, and dozens of other applications that use video and audio decoders in Linux. Fortunately quite a lot of the world is not the United States, and today VLC is the world's second most popular media player and has never paid one cent in patent licensing fees. But of course, yes, being open source does not magically exempt you from patent laws in countries with insane, broken patent laws. In practice, if you want to make a large-scale commercial application that will be distributed in the US that uses x264, you will probably need to pay for an MPEG-LA license. They're quite cheap, though: 0 cents per unit up to 100k units, 20 cents per unit after that until 5 million, and 10 cents per unit after that. Largely, the question of whether a distro contains any particular piece of software is whether the people who run the repositories are willing to host it. This applies both to possibly-patented software, but also to libraries like DeCSS (necessary to play DVDs) that violate the laws in some countries, but not others.
    - shaver 15y ago
      You're right, of course. While I believe that the MPEG-LA's enforcement practices are not non-discrimatory as it's claimed to be, that's a different discussion, and it wasn't appropriate to single x264 out that way. I apologize.
devongovett 15y ago
Here's the BadassJS perspective on all of this very impressive stuff! http://badassjs.com/post/12035631618/broadway-an-h-264-decoder-in-javascript-running-at http://badassjs.com/post/12035631618/broadway-an-h-264-decod...
andhow 15y ago
Wow. 45 fps in a Firefox nightly, 20 fps in Chromium. Pretty impressive for JS.
- mbebenita 15y ago
  Give it another try. We made a few small improvements that turned out to be quite significant.

142 points by tnajdek 15y ago

Running on a macbook pro with i5 dual core 2.4 ghz, it uses 1 core at 100% in firefox and manages: 1.7 fps It's cool, but I don't see any real world applications. Anyone got any ideas?

ck2 15y ago
I think it's more of an (impressive) mental exercise - if the code can be made straightforward enough to run on javascript, the future applications can only grow from there.
- robert-boehnke 15y ago
  It's the Android encoder written in c++ cross compiled to JavaScript, not much of a mental exercise. Not to say it's not cool :)
  - ZeroGravitas 15y ago
    They have a hand-coded javascript version in progress too: https://github.com/mbebenita/Broadway/tree/master/Play https://github.com/mbebenita/Broadway/tree/master/Play
salmanapk 15y ago
Strange.. It was running at 4 fps on my Pentium 4 lol
- hmottestad 15y ago
  hmmm.... What browser, OS and what Pentium 4 processor? And for how long did you run the video? Mine drops to 1.7 after a few seconds, the first few frames are dark so in the beginning I get 6 fps.
  - salmanapk 15y ago
    Nightly, XP, 2.4 Ghz. Nearly 5 Minutes.. Here it starts at like 0.6 fps, and then runs at 4-7 fps onwards.
    - hmottestad 15y ago
      Must be something with Nightly then, I'm just running the regular version. Says a lot about the importance of software over hardware :)
      - shaver 15y ago
        It definitely needs nightly. I'm not sure what's up with salmanapk's setup, will need to dig into it.
shaver 15y ago
You need Firefox nightly for it to run well, because it depends on the hybrid static/dynamic type inference work and some other optimizations. I believe that Brendan was demoing it from a Mac laptop of some kind. As far as applications, it will let content providers who want to ship H.264 and pay the license fees do so without requiring all browser developers to pay such fees. (For Firefox it would be a meaningful portion of Mozilla's engineering budget.) And it's a pretty compelling demo, IMO, of the fact that we're not done with JS performance yet, and that people don't need to be running to Dart or NaCl or other rip-and-replace technologies in order to get great perf. There's a lot of opportunity for even better performance on the engine side as well as the library side: they started a week ago with emscripten, Closure and the libstagefright C sources...
grayrest 15y ago
With today's Nightly I'm getting ~20fps on a 3.4 GHz Core i7 iMac but with drops into the 7fps range on some video transitions. Surprisingly, I get roughly the same perf with a current Aurora as well but the drops only go down to ~12fps. Trying it on the released Firefox gets around 2fps.
azakai 15y ago
1. This works much faster on Firefox nightly (that's where we demo'd it on). Nightly has a lot of JS engine improvements that are not in stable yet (but will be in a month or two). 2. This does not use any hardware acceleration yet. This is simply compiling the Android C decoder into JS, nothing else - just a few days of work. We will now start to look at actually optimizing the code specifically for JS, and also to use GPU shaders for the relevant part of the code. Both of those can potentially make this even faster. Regarding real-world applications, the interesting possibility is for websites to ship their own codecs. As I mentioned in a comment above, imagine an anime video website that ships a video decoder optimized for that kind of content - it could be much more efficient than stock H.264. Also, if websites can ship their own decoders, they can continually improve them (unlike now where websites rely on client decoders which are not constantly being improved).

__alexs 15y ago

Apparently this can actually get decent frame rates. Although I suspect they are using the patched version of the JS interpreter mentioned on github. http://yfrog.com/nmng0z http://yfrog.com/nmng0z Still not sure how this is meant to actually be useful though. The problem with H.264 isn't availability of implementations, it's being non-free and heavily patented. I can kind of see a use for this if you are a big content provider with a bunch of cash and want to distribute video to Firefox users without transcoding but it still seems pretty derpy.

shaver 15y ago
Brendan's demo was running at 30fps, I believe on a Macbook Pro (I'll find out). Some content will run slower right now, which is true of all codecs AFAIK but is more so for this stuff right now. The patches linked from the github README aren't necessary to run the transpiled version that was in the demo -- it's a memory optimization that's being used in the tuned-for-JS version. As regards derpiness: it lets content distributors decide to pay for H.264, if they want, exactly, and paves the way for other codecs to be deployed by such distributors as well. It also runs the codec in a managed environment -- format decoders are often very fertile territory for exploitable bugs, since they are pretty much by definition all about pointer math and byte-poking. But the initial intent, when they decided a week ago to try it, was to push the envelope of JS performance such that we find new ways to extend said envelope.
- pcwalton 15y ago
  It was on a MacBook Pro. We tried it on a MacBook Air too and it performed reasonably well.
  - watmough 15y ago
    My 2007 MBP manages a decent 20 fps but starts pretty slow and averages 15. That's a 2.4GHz C2D. Need to try it on my i52400.
    - mbebenita 15y ago

hackermom 15y ago

Funny hobby project.

BrendanEich 15y ago
... said the dinosaur to the first mammal, re: size, fur, and all that hobby fluff.
- hackermom 15y ago
  You just can't compare a JS library for h.264 decoding with ditto made in C. The latter is usable while the former is practically useless even for an extreme case of f.e. online, browser-based video editing. What is the point you are trying to make?

gerggerg 15y ago

Any one know what the legal implications of using a free decoder for a patented compression format are? Products like flash make it seem free as Adobe pays a flat rate to the patent holders for decoding mp3, h.264, etc. alleviating their users from having to worry about royalties. But would using this decoder, though free and open source, be infringing a patent? note: I realize it's not production code. I'm asking more about the concept.

shaver 15y ago
Use and distribution of implementations requires a license (though, curiously, x264 seems to get a pass), so using the decoder will require a license as with using a closed-source one.
- DarkShikari 15y ago
  (though, curiously, x264 seems to get a pass) If x264 does, so does VLC, mplayer, ffmpeg, gstreamer, and dozens of other applications that use video and audio decoders in Linux. Fortunately quite a lot of the world is not the United States, and today VLC is the world's second most popular media player and has never paid one cent in patent licensing fees. But of course, yes, being open source does not magically exempt you from patent laws in countries with insane, broken patent laws. In practice, if you want to make a large-scale commercial application that will be distributed in the US that uses x264, you will probably need to pay for an MPEG-LA license. They're quite cheap, though: 0 cents per unit up to 100k units, 20 cents per unit after that until 5 million, and 10 cents per unit after that. Largely, the question of whether a distro contains any particular piece of software is whether the people who run the repositories are willing to host it. This applies both to possibly-patented software, but also to libraries like DeCSS (necessary to play DVDs) that violate the laws in some countries, but not others.
  - shaver 15y ago
    You're right, of course. While I believe that the MPEG-LA's enforcement practices are not non-discrimatory as it's claimed to be, that's a different discussion, and it wasn't appropriate to single x264 out that way. I apologize.

devongovett 15y ago

Here's the BadassJS perspective on all of this very impressive stuff! http://badassjs.com/post/12035631618/broadway-an-h-264-decoder-in-javascript-running-at http://badassjs.com/post/12035631618/broadway-an-h-264-decod...

andhow 15y ago

Wow. 45 fps in a Firefox nightly, 20 fps in Chromium. Pretty impressive for JS.

mbebenita 15y ago
Give it another try. We made a few small improvements that turned out to be quite significant.