30 ms·

A Simple Explanation: VLC.js

135 points by vmorgulis 10y ago

bambax 10y ago
But wouldn't a browser plugin be easier to develop and also easier to use for the end user? Exactly what is gained by having it run in js? (Better portability maybe? But VLC already runs on almost all platforms natively...?)
- voltagex_ 10y ago
  Because the user doesn't have to install anything.
- coldtea 10y ago
  >But wouldn't a browser plugin be easier to develop and also easier to use for the end user? The trend for the last couple of years or so has been for browsers to deprecate third-party browser plugins.
- fridek 10y ago
  Because every browser has different plugins and their concept tends to change every few years, for good (security) or bad (politics) reasons. Once VLC.js runs, you can reasonably expect it to run forever and be just another compilation target.
  - throwanem 10y ago
    Exactly. For example, Mozilla is deprecating NPAPI just as fast as it can, because it long predates e10s (Chrome-style process isolation) and they do not play well together. e10s has been so long delayed already, and Firefox's performance problems are so intractable to solve without it, that from the perspective of a very long-time Firefox user like me this is very obviously the correct course of action. This deprecation will break the existing VLC plugin for Firefox, but that's not a huge deal because it never has worked very well in any case - developing a reliable NPAPI plugin is a seriously hard problem in general and has been since browsers got complicated enough to do interesting things in the first place. You also have to restart the browser to make any changes to the set of installed NPAPI plugins, and that's a lot more of a problem than it was before the browser became an application runtime. Now that the JS runtime is fast and capable enough to run "real application" code, we derive a much better experience by leveraging that power than we do by targeting a legacy interface that is relatively ill documented and much easier to get wrong in ways that break the whole browser rather than just one page or tab. We also get cross-browser compatibility of a sort that NPAPI never could offer. I get that a lot of people think it's weird for the web browser's page scripting language, which was nothing more than a cute toy a couple decades ago, to be eating the world like this. I think it's weird, and it's how I earn a living! But, weird or not, it's also the best application platform we have or likely soon will have for a broad and widening variety of use cases, and that is because, while no more perfect than anything else, it actually works really well. Railing bitterly against technology that works really well seems potentially counterproductive to me. But I appreciate there exists a diversity of opinion on the subject.
  - acqq 10y ago
    > Once VLC.js runs, you can reasonably expect it to run forever You can reasonably expect it to run... until the first non-backward compatible change in anything the code depends on. The major assumption for this to keep working is that everything has to be permanently kept up-to-date for all the breaking dependencies. And I assume it will always be easier to produce a Linux or Windows binary that works relatively long without changes than anything that non-trivially depends on the web technologies. A few years ago, as far as I know, it was still possible to run the Netscape Navigator binary which was around 20 years old on the recent Linux kernel.
- sakri 10y ago
  Killing the flash player plugin for video with another plugin?
- fulafel 10y ago
  Security is one good reason: nearly all native media players (including VLC[1]) are full of serious security vulnerabilities that let people take over your computer via crafted video files. [1] http://www.cvedetails.com/product/9978/Videolan-Vlc-Media-Player.html?vendor_id=5842 http://www.cvedetails.com/product/9978/Videolan-Vlc-Media-Pl...
kowdermeister 10y ago
> I firmly believe that this project, fundamentally, would change the relationship of audio/video to the web. Aren't they banging open doors? HTML5 video and audio is great and with WebAudio and WebGL you can manipulate every aspect of the experience. EQ, filters are possible. What else do you need? Subtitles? I just don't understand what problem are they trying to solve because "I can't run VLC in the browser" certainly isn't one.
- grenoire 10y ago
  You can even get subtitles on top of HTML5 videos with simple positioned elements with text shadows for borders. I can't say I understand the motive of the project either, other than "see, everything can be done in JavaScript!"
  - danielsamuels 10y ago
    Or just use a `<track kind="subtitles">` and reference a vtt file?
    - fridek 10y ago
      Which is a format you need to convert to rather than use existing srt or anything else really.
    - kuschku 10y ago
      Which doesn't allow SRT subtitles, or XML subtitles (hello German public broadcasters!), or ASS subtitles (hello anime fandub groups!) Doing this natively in JS via canvas allows you exactly that - rendering these things natively, in real time, with all the support for rare features you could ever need.
  - vdnkh 10y ago
    Much harder than it sounds if you care about live streaming, video playlists, and cross-browser support. Don't forget to mention the 3 big subtitles formats (608/708/VTT) and some smaller ones (SRT/DFXP). Also, here's a fun quirk of the subtitles track element: you cannot remove a track once added. Another obstacle for full FCC compliance: styling. You must allow users to style your captions. Captions are put in the shadow DOM - Firefox does not allow you to style the shadow dom. Don't forget to add IE into the mix!
antome 10y ago
Does this mean that developers will go as far as maintaining a port of FFMPEG to JavaScript? Being able to decode 10bit videos in pure JS sounds both insane and insanely cool
- acqq 10y ago
  > a port of FFMPEG to JavaScript It already exists, see my other comment for more details.
  - kinlan 10y ago
    There are a number of ports powered by Emscripten (https://github.com/Kagami/ffmpeg.js https://github.com/Kagami/ffmpeg.js) I wrote about some stuff here https://paul.kinlan.me/ffmpeg-ideas/ https://paul.kinlan.me/ffmpeg-ideas/ when building https://paulkinlan.github.io/deviceframe.es/ https://paulkinlan.github.io/deviceframe.es/ (I do a lot of screencasts and need a simple way to scale this out to other people that doesn't involve installing ffmpeg and scripting) EDIT and Note: Also I just saw your comment about videoconvert.js
    - acqq 10y ago
      Do I understand it correctly that you want the users to do javascript ffmpeg processing on their android devices? I admit I have no idea what deviceframe is, so I'd also like to read some explanation on that too. Have you compared the speed of Android javascript ffmpeg vs PC native? What happens with the battery?
      - lawik 10y ago
        Javascript and as soon as possible WebAssembly, as I understand it. I think mobile is a secondary consideration in this at this point. Considering it is connected to the internet archival efforts and simply ensuring the survival of numerous video/audio formats I imagine they are fine if it isn't efficient but simply effective at this point. Edit: on second thought, you might not be commenting on the VLC.js thing, I haven't read about the ffmpeg js stuff ;)
acqq 10y ago
From the previous discussion: vespakoen noted that there is already ffmpeg library compiled with emscripten: http://bgrins.github.io/videoconverter.js/demo http://bgrins.github.io/videoconverter.js/demo My speed test of webm to mp4 conversion (it doesn't display the video, it's just ffmpeg) showed the JS version around 10 times slower than the native ffmpeg. The test was on relatively recent i7 and the latest Firefox (certainly having as fast asm.js as possible) then (around a month ago). I haven't investigated the cause for such significant speed difference.
- colek42 10y ago
  I have a project that decodes mpegts packets and renders the frames to a webGL canvas using FFMpeg. https://github.com/colek42/streamingDemo https://github.com/colek42/streamingDemo. Full disclosure much of the client code is copy/pasta from https://github.com/opensensorhub https://github.com/opensensorhub Edit: add attribution
  - rogerdpack 10y ago
    is there a live demo some/anywhere?
    - colek42 10y ago
      Sorry, I am using it to proxy UDP streams which requires a backend component due to browser restrictions. If I hosted a demo it may end up costing a nontrivial amount of money. I am developing this closed source for my employer, but if there is enough interest I can push some of the changes to the open source repo.`
andrewvijay 10y ago
This is very exciting and to a lot of people, doing things with only a browser is so much enabling!
sandGorgon 10y ago
This is one of the reason I'm truly excited about the future of javascript in its ES6 or Typescript form. If they manage to make this run and use emscripten and manage to build all the video computation optimizations.. how far of a stretch is it really to combine lodash+emscripten+BLAS/LAPACK to build a high performance computing platform based on javascript ?
- na85 10y ago
  Honest question: Why is building an HPC platform on a language designed in less than 2 weeks something to aspire to? Why not aspire to replace javascript entirely with something better?
  - sandGorgon 10y ago
    huh? ES6/emscripten/Typescript was not designed in 2 weeks. could you talk about what you mean.
    - Outpox 10y ago
      He's talking about the first JS specification which was done in 2 weeks IIRC. I've seen such comments several times which completely discard all the effort made to improve JS until now.
    - lsadam0 10y ago
      He's talking about Javascript itself. The language has a ton of short-comings, hence the rise of languages that transpile to JS. I agree with the previous sentiment, we should replace JS with something better. JS is a nightmare.
    - goatlover 10y ago
      The first version of the language, upon which all other versions have been based, was done in 2 weeks. There are fundamental shortcomings in JS that exist to this day because of that, and because the web has to remain backwards compatibility. So, despite all the improvements over the year, JS is still poor in some areas, particularly when you're talking about porting applications to JS. It was never designed for that.
bmarkovic 10y ago
Wouldn't webxmp (also emscripten based) be a better match for MOD/S3M/XM support: http://www.wothke.ch/webxmp/ http://www.wothke.ch/webxmp/
- spacemanmatt 10y ago
  That's really neat! I still have a few music modules I can't part with.
kbutler 10y ago
Power consumption and performance are big concerns, because JavaScript (software) decoders can't leverage hardware acceleration. But archive.org wants to preserve and publish many formats that are not high resolution, high frame-rate video, and we can expect both performance and power consumption to improve over time.
- _joel 10y ago
  HTML5 canvas can use hardware acceleration, of course. If the codec being used is suitable for HW acceleration (i.e. h/x624 yes, VP9 no - generally)
zaphar 10y ago
At what point will the current trend to treat the browser and WebAssembly as a compile target for traditionally fat client applications raise the need for a browser package manager ala Apt or Yum? When will dynamic linking become a necessity because due to memory usage, disk space, and patches for 0day exploits in upstream libraries? It seems like the more the browser takes on the role of an OS the more it will need all the stuff that OS's have built up over the years to make it manageable.
- pasta 10y ago
  And the strange thing is: I can already start a VLC binary over the network via my file manager. A lot of people think we need to start rethinking the web. But as the poster points out: "but that’s what we have and here we go"
  - zaphar 10y ago
    Yeah at some point we by default decided the benefits of just go to this URL as a way to install software outweighed the negatives. It wasn't even really a deliberate decision collectively. We solved problems in a locally optimal way but I suspect it's not the globally optimal solution.
- jsight 10y ago
  That is kind of the role filled by "npm" and "yarn". They are obviously a lot different from OS package managers, but ultimately they solve a parallel type of problem for the web.
  - zaphar 10y ago
    Actually no. Those tools are development package managers not user package managers. Two very different use cases. A development package managers is part of how the developer builds the application. A user package manager is how the consumer of the application installs it while also managing all the shared libraries and installing security updates for each of the pieces.

135 points by vmorgulis 10y ago

But wouldn't a browser plugin be easier to develop and also easier to use for the end user? Exactly what is gained by having it run in js? (Better portability maybe? But VLC already runs on almost all platforms natively...?)

voltagex_ 10y ago
Because the user doesn't have to install anything.
coldtea 10y ago
>But wouldn't a browser plugin be easier to develop and also easier to use for the end user? The trend for the last couple of years or so has been for browsers to deprecate third-party browser plugins.
fridek 10y ago
Because every browser has different plugins and their concept tends to change every few years, for good (security) or bad (politics) reasons. Once VLC.js runs, you can reasonably expect it to run forever and be just another compilation target.
- throwanem 10y ago
  Exactly. For example, Mozilla is deprecating NPAPI just as fast as it can, because it long predates e10s (Chrome-style process isolation) and they do not play well together. e10s has been so long delayed already, and Firefox's performance problems are so intractable to solve without it, that from the perspective of a very long-time Firefox user like me this is very obviously the correct course of action. This deprecation will break the existing VLC plugin for Firefox, but that's not a huge deal because it never has worked very well in any case - developing a reliable NPAPI plugin is a seriously hard problem in general and has been since browsers got complicated enough to do interesting things in the first place. You also have to restart the browser to make any changes to the set of installed NPAPI plugins, and that's a lot more of a problem than it was before the browser became an application runtime. Now that the JS runtime is fast and capable enough to run "real application" code, we derive a much better experience by leveraging that power than we do by targeting a legacy interface that is relatively ill documented and much easier to get wrong in ways that break the whole browser rather than just one page or tab. We also get cross-browser compatibility of a sort that NPAPI never could offer. I get that a lot of people think it's weird for the web browser's page scripting language, which was nothing more than a cute toy a couple decades ago, to be eating the world like this. I think it's weird, and it's how I earn a living! But, weird or not, it's also the best application platform we have or likely soon will have for a broad and widening variety of use cases, and that is because, while no more perfect than anything else, it actually works really well. Railing bitterly against technology that works really well seems potentially counterproductive to me. But I appreciate there exists a diversity of opinion on the subject.
- acqq 10y ago
  > Once VLC.js runs, you can reasonably expect it to run forever You can reasonably expect it to run... until the first non-backward compatible change in anything the code depends on. The major assumption for this to keep working is that everything has to be permanently kept up-to-date for all the breaking dependencies. And I assume it will always be easier to produce a Linux or Windows binary that works relatively long without changes than anything that non-trivially depends on the web technologies. A few years ago, as far as I know, it was still possible to run the Netscape Navigator binary which was around 20 years old on the recent Linux kernel.
sakri 10y ago
Killing the flash player plugin for video with another plugin?
fulafel 10y ago
Security is one good reason: nearly all native media players (including VLC[1]) are full of serious security vulnerabilities that let people take over your computer via crafted video files. [1] http://www.cvedetails.com/product/9978/Videolan-Vlc-Media-Player.html?vendor_id=5842 http://www.cvedetails.com/product/9978/Videolan-Vlc-Media-Pl...

kowdermeister 10y ago

> I firmly believe that this project, fundamentally, would change the relationship of audio/video to the web. Aren't they banging open doors? HTML5 video and audio is great and with WebAudio and WebGL you can manipulate every aspect of the experience. EQ, filters are possible. What else do you need? Subtitles? I just don't understand what problem are they trying to solve because "I can't run VLC in the browser" certainly isn't one.

grenoire 10y ago
You can even get subtitles on top of HTML5 videos with simple positioned elements with text shadows for borders. I can't say I understand the motive of the project either, other than "see, everything can be done in JavaScript!"
- danielsamuels 10y ago
  Or just use a `<track kind="subtitles">` and reference a vtt file?
  - fridek 10y ago
    Which is a format you need to convert to rather than use existing srt or anything else really.
  - kuschku 10y ago
    Which doesn't allow SRT subtitles, or XML subtitles (hello German public broadcasters!), or ASS subtitles (hello anime fandub groups!) Doing this natively in JS via canvas allows you exactly that - rendering these things natively, in real time, with all the support for rare features you could ever need.
- vdnkh 10y ago
  Much harder than it sounds if you care about live streaming, video playlists, and cross-browser support. Don't forget to mention the 3 big subtitles formats (608/708/VTT) and some smaller ones (SRT/DFXP). Also, here's a fun quirk of the subtitles track element: you cannot remove a track once added. Another obstacle for full FCC compliance: styling. You must allow users to style your captions. Captions are put in the shadow DOM - Firefox does not allow you to style the shadow dom. Don't forget to add IE into the mix!

antome 10y ago

Does this mean that developers will go as far as maintaining a port of FFMPEG to JavaScript? Being able to decode 10bit videos in pure JS sounds both insane and insanely cool

acqq 10y ago
> a port of FFMPEG to JavaScript It already exists, see my other comment for more details.
- kinlan 10y ago
  There are a number of ports powered by Emscripten (https://github.com/Kagami/ffmpeg.js https://github.com/Kagami/ffmpeg.js) I wrote about some stuff here https://paul.kinlan.me/ffmpeg-ideas/ https://paul.kinlan.me/ffmpeg-ideas/ when building https://paulkinlan.github.io/deviceframe.es/ https://paulkinlan.github.io/deviceframe.es/ (I do a lot of screencasts and need a simple way to scale this out to other people that doesn't involve installing ffmpeg and scripting) EDIT and Note: Also I just saw your comment about videoconvert.js
  - acqq 10y ago
    Do I understand it correctly that you want the users to do javascript ffmpeg processing on their android devices? I admit I have no idea what deviceframe is, so I'd also like to read some explanation on that too. Have you compared the speed of Android javascript ffmpeg vs PC native? What happens with the battery?
    - lawik 10y ago
      Javascript and as soon as possible WebAssembly, as I understand it. I think mobile is a secondary consideration in this at this point. Considering it is connected to the internet archival efforts and simply ensuring the survival of numerous video/audio formats I imagine they are fine if it isn't efficient but simply effective at this point. Edit: on second thought, you might not be commenting on the VLC.js thing, I haven't read about the ffmpeg js stuff ;)

acqq 10y ago

From the previous discussion: vespakoen noted that there is already ffmpeg library compiled with emscripten: http://bgrins.github.io/videoconverter.js/demo http://bgrins.github.io/videoconverter.js/demo My speed test of webm to mp4 conversion (it doesn't display the video, it's just ffmpeg) showed the JS version around 10 times slower than the native ffmpeg. The test was on relatively recent i7 and the latest Firefox (certainly having as fast asm.js as possible) then (around a month ago). I haven't investigated the cause for such significant speed difference.

colek42 10y ago
I have a project that decodes mpegts packets and renders the frames to a webGL canvas using FFMpeg. https://github.com/colek42/streamingDemo https://github.com/colek42/streamingDemo. Full disclosure much of the client code is copy/pasta from https://github.com/opensensorhub https://github.com/opensensorhub Edit: add attribution
- rogerdpack 10y ago
  is there a live demo some/anywhere?
  - colek42 10y ago
    Sorry, I am using it to proxy UDP streams which requires a backend component due to browser restrictions. If I hosted a demo it may end up costing a nontrivial amount of money. I am developing this closed source for my employer, but if there is enough interest I can push some of the changes to the open source repo.`

andrewvijay 10y ago

This is very exciting and to a lot of people, doing things with only a browser is so much enabling!

sandGorgon 10y ago

This is one of the reason I'm truly excited about the future of javascript in its ES6 or Typescript form. If they manage to make this run and use emscripten and manage to build all the video computation optimizations.. how far of a stretch is it really to combine lodash+emscripten+BLAS/LAPACK to build a high performance computing platform based on javascript ?

na85 10y ago
Honest question: Why is building an HPC platform on a language designed in less than 2 weeks something to aspire to? Why not aspire to replace javascript entirely with something better?
- sandGorgon 10y ago
  huh? ES6/emscripten/Typescript was not designed in 2 weeks. could you talk about what you mean.
  - Outpox 10y ago
    He's talking about the first JS specification which was done in 2 weeks IIRC. I've seen such comments several times which completely discard all the effort made to improve JS until now.
  - lsadam0 10y ago
    He's talking about Javascript itself. The language has a ton of short-comings, hence the rise of languages that transpile to JS. I agree with the previous sentiment, we should replace JS with something better. JS is a nightmare.
  - goatlover 10y ago
    The first version of the language, upon which all other versions have been based, was done in 2 weeks. There are fundamental shortcomings in JS that exist to this day because of that, and because the web has to remain backwards compatibility. So, despite all the improvements over the year, JS is still poor in some areas, particularly when you're talking about porting applications to JS. It was never designed for that.

bmarkovic 10y ago

Wouldn't webxmp (also emscripten based) be a better match for MOD/S3M/XM support: http://www.wothke.ch/webxmp/ http://www.wothke.ch/webxmp/

spacemanmatt 10y ago
That's really neat! I still have a few music modules I can't part with.

kbutler 10y ago

Power consumption and performance are big concerns, because JavaScript (software) decoders can't leverage hardware acceleration. But archive.org wants to preserve and publish many formats that are not high resolution, high frame-rate video, and we can expect both performance and power consumption to improve over time.

_joel 10y ago
HTML5 canvas can use hardware acceleration, of course. If the codec being used is suitable for HW acceleration (i.e. h/x624 yes, VP9 no - generally)

zaphar 10y ago

At what point will the current trend to treat the browser and WebAssembly as a compile target for traditionally fat client applications raise the need for a browser package manager ala Apt or Yum? When will dynamic linking become a necessity because due to memory usage, disk space, and patches for 0day exploits in upstream libraries? It seems like the more the browser takes on the role of an OS the more it will need all the stuff that OS's have built up over the years to make it manageable.

pasta 10y ago
And the strange thing is: I can already start a VLC binary over the network via my file manager. A lot of people think we need to start rethinking the web. But as the poster points out: "but that’s what we have and here we go"
- zaphar 10y ago
  Yeah at some point we by default decided the benefits of just go to this URL as a way to install software outweighed the negatives. It wasn't even really a deliberate decision collectively. We solved problems in a locally optimal way but I suspect it's not the globally optimal solution.
jsight 10y ago
That is kind of the role filled by "npm" and "yarn". They are obviously a lot different from OS package managers, but ultimately they solve a parallel type of problem for the web.
- zaphar 10y ago
  Actually no. Those tools are development package managers not user package managers. Two very different use cases. A development package managers is part of how the developer builds the application. A user package manager is how the consumer of the application installs it while also managing all the shared libraries and installing security updates for each of the pieces.