5 ms·

Show HN: Video Transcoder on the browser using WASM

cryogenicplanet 6y ago
Hello HN Modfy is a purely browser-based privacy-first video tool capable of performing tasks like converting, compression, etc without uploading your files. I thought it would be cool to process video in the browser using Web Assembly and FFmpeg The project is also open source and based on https://github.com/ffmpegwasm/ffmpeg.wasm https://github.com/ffmpegwasm/ffmpeg.wasm Let me know what you think!
- untog 6y ago
  How does the performance compare to running ffmpeg ad usual? I’m guessing that WASM means you don’t have access to things like hardware acceleration?
  - cryogenicplanet 6y ago
    It is definitely slower than using ffmpeg normally, but the idea is for more non technical users it adds convenience and they don't have to use a command line. Yeah this version doesn't have hardware acceleration, but if I am not wrong WASM has hardware acceleration in development or coming soon.
    - sktrdie 6y ago
      Awesome work! The easy-of-use and privacy capabilities of this totally out-weight the somewhat slower performance. The safety one has with being able to do this in the browser, rather than having to download a tool that is not "sandboxed" and is capable of potentially running anything on your system, is amazing. Regarding the slowness, can you comment on why you think WebAssembly is slower than native? I'm sure some remarks would be very helpful to the Wasm team. Thanks!
  - whyleyc 6y ago
    I tested with a 64MB, 124s long, 1152x720 .MOV video converted to .MP4 and here's what I saw: modfy.video - 258 seconds native ffmpeg - 23 seconds This was using a 2020 MBP with a 2.3 GHz Quad-Core Intel Core i7 processor & 32GB of RAM. My local ffmpeg version was v4.3.1. So it looks like performance definitely takes a hit due to WASM, but I'm sure performance will improve, and this is technically a very impressive achievement. The UI is really nice, and the privacy benefits of remaining client side, whilst delivered through the Web are great (i.e. nothing to install locally). I'm one of the co-founders @ Zamzar so really interested in this space. Congrats to @cryogenicplanet on this project - you've done a great job with it!
    - the-dude 6y ago
      Isn't .MOV already .MP4?
      - dylan604 6y ago
        No. MP4 is based on MOV. MP4 is typically going to have h.264 video with AAC audio. MOV is not limited in video codec, nor audio codec. Even though MediaInfo reports MOV as MPEG-4, they are not the same.
        Denvercoder9 6y ago
        MP4 isn't limited in video codec or audio codec either, and it's quite common to have non-H.264 or non-AAC codecs in them. The only difference between MOV and MP4 are a few later created extensions, and the difference in every day use is negligble.
        dylan604 6y ago
        MP4 is an ISO standardized version of where MOV was at the time it was made a standard. MOV is an Apple product that they are free to update as they see fit. Apple has since modified the QT container regarding things like the moov atoms for aspect ratio, color systems (709 vs 609 etc), and all of the other similar fiddly bits. This is the major differences between the two to me.
- ArtWomb 6y ago
  Thanks for building, cryogenicplanet! Just wanted to add I really like the "no-upload" programming paradigm a lot. I use browser native webgl rendering internally for example to automate generation of static digital art assets. All samples and finished images are transferred via http. And its easy to run on any device: just fire up chrome and enter url. I'd like to do the same for the video asset generation pipeline as well. For which I do most ffmpeg processing on host. WASM build allows for browser based video generation on the fly ;)
  - cryogenicplanet 6y ago
    Yeah this is definitely try to push that dream of not having to upload/download software and doing most if not all things on the browser itself
- FraKtus 6y ago
  I am not knowledgeable with WASM, can it enable threading in FFmpeg? That’s where FFmpeg shine...
  - markdog12 6y ago
    Yes, SIMD as well, although they're not available on all browsers/devices.
- leptons 6y ago
  I'm curious about using this to transcode youtube videos in the browser. Wondering if CORS is going to be an issue. Looks like I found another rabbit hole to keep me occupied for a while.
kevincox 6y ago
This is a real cool tool and I would love to see more simple tools client-side only. However I have serious concerns about the performance. When developing a solver (https://ricochetrobots.kevincox.ca/ https://ricochetrobots.kevincox.ca/) I found that compared to the native performance running it in WASM was ~2x slower. (Running it on a phone compared to a laptop was another ~2x). However I doubt the native compiler was using large amounts of SIMD. IIUC video encoding often will use a large amount of SIMD instructions so I would expect the slowdown to be worse. On the other hand SIMD support in WASM is in progress: https://www.chromestatus.com/feature/6533147810332672 https://www.chromestatus.com/feature/6533147810332672 (preview available on chromium)
- londons_explore 6y ago
  I too would like to know if the performance shortcomings of webassembly are inherent to its design (ie. The sandboxing mechanism), or are just due to a poor implementation. Basic things like not being able to get stuff from CPU to screen without multiple memcpy's are all problematic too...
  - oscargrouch 6y ago
    Not a compiler expert, but looking at the bytecode it looked fairly limited and minimal number of opcodes. Giving it has to be universal, i imagine where a LLVM bitcode op might translate to 1, 2 or 3 native opcodes, with WebAssembly you might with much more opcodes in the native output. I mean, of course they will be able to make it better with time and effort, but i dont think we can spect a much bigger speedup in comparison with the native equivalent. Note: this is all very speculative, but by looking at it, it looked great as an accelerator, specially for math stuff, but i imagine that it will be hard even to beat the Javascript code for generic stuff giving how much it was already optimized for it. If any compiler expert here that is familiar with it want to correct in my assumptions, please feel free to do it. (Not event touching in issues like the ones you mention, like memory copy, where the compiler cannot translate the thing in a more optmized manner giving it cannot speculate about the environment and optimize for it)
- cryogenicplanet 6y ago
TorchAtlas 6y ago
This is very interesting! Would definitely be useful non-technical users. I have a few UX concerns, though. - By default, it's set to All Features instead of Basic Features - The app repeatedly says it doesn't upload your files, yet there's a big button in the app saying Upload File. Perhaps "Your files are never uploaded" should be reworded to something like "Your files never leave your device"? - It also was not obvious how to get started with the app. Perhaps some text could be added to the play button?
- dylan604 6y ago
  Or change the text on the button. 'Add Files', 'Import Files', etc would be equally useful while not confusing.
- cryogenicplanet 6y ago
  Thanks for the feedback, will definitely take this into account and iterate on this feedback. Someone opened an issue related to this too, so will update it on that issue https://github.com/Etwas-Builders/modfy.video/issues/88 https://github.com/Etwas-Builders/modfy.video/issues/88
jszymborski 6y ago
Wow! I wish I saw this a couple of months ago when I set up server-side transcoding for a client ;_;
deleted 6y ago
[deleted]
hardwaregeek 6y ago
I tried running something like this on my own, but I got OOM errors pretty quickly. 4 gigs isn't a lot when it comes to video processing. Hopefully wasm64 and WebGPU will make it a lot better.
- cryogenicplanet 6y ago
  I think this is as a result of the video have to be uploaded into RAM to be processed, this is definitely something I want to try to fix where I access the videos on the native file system to allow much larger files
cbhl 6y ago
On the left side, the label says "Click or Drag to upload", and at the bottom there is a label saying "Your files are not uploaded anywhere". If you have control over the left file picker, it might make sense to change the label to "Click or Drag to pick a video" or similar. Really great to see this -- I once worked on a cloud-based video editor and the cost of running ffmpeg in the cloud was surprisingly prohibitive.
- cryogenicplanet 6y ago
  Thanks! Already tracking the label issue here https://github.com/Etwas-Builders/modfy.video/issues/88 https://github.com/Etwas-Builders/modfy.video/issues/88 will resolve that tonight I think that's another great thing for me with this product, is that it almost costs nothing to run it (Thanks netlify)
alfg 6y ago
This is pretty cool. I've also been playing around in this area lately and noticed there's still a drastic performance hit. But seems to work great for smaller files. Question: Are you compiling FFmpeg to WASM, or FFMpeg's libav and interfacing with that directly? Also for anyone curious, the author of ffmpeg.wasm[1] has an excellent guide on compiling FFmpeg to WASM: https://itnext.io/build-ffmpeg-webassembly-version-ffmpeg-js-part-1-preparation-ed12bf4c8fac https://itnext.io/build-ffmpeg-webassembly-version-ffmpeg-js... [1] https://github.com/ffmpegwasm/ffmpeg.wasm https://github.com/ffmpegwasm/ffmpeg.wasm
ggonweb 6y ago
Is there a restriction on the size of the file or can it save the contents on the fly to disk like streamsaver?
- cryogenicplanet 6y ago
  Currently there is a restriction on size of file because it has to process the video in your memory and isn't saving content on the fly to disk but that is definitely something I can try to add moving forward
  - ggonweb 6y ago
    Good job on your release, Curious, How is your version different from kagami or videoconverter versions, which were file conversions and they did not have streaming options, even though they had some patches who showed how to do it. Does yours support streaming options so that the streams can be saved to file serially using streamsaver.js.
Akaolisa 6y ago
Very nice

Hello HN Modfy is a purely browser-based privacy-first video tool capable of performing tasks like converting, compression, etc without uploading your files. I thought it would be cool to process video in the browser using Web Assembly and FFmpeg The project is also open source and based on https://github.com/ffmpegwasm/ffmpeg.wasm https://github.com/ffmpegwasm/ffmpeg.wasm Let me know what you think!

untog 6y ago
How does the performance compare to running ffmpeg ad usual? I’m guessing that WASM means you don’t have access to things like hardware acceleration?
- cryogenicplanet 6y ago
  It is definitely slower than using ffmpeg normally, but the idea is for more non technical users it adds convenience and they don't have to use a command line. Yeah this version doesn't have hardware acceleration, but if I am not wrong WASM has hardware acceleration in development or coming soon.
  - sktrdie 6y ago
    Awesome work! The easy-of-use and privacy capabilities of this totally out-weight the somewhat slower performance. The safety one has with being able to do this in the browser, rather than having to download a tool that is not "sandboxed" and is capable of potentially running anything on your system, is amazing. Regarding the slowness, can you comment on why you think WebAssembly is slower than native? I'm sure some remarks would be very helpful to the Wasm team. Thanks!
- whyleyc 6y ago
  I tested with a 64MB, 124s long, 1152x720 .MOV video converted to .MP4 and here's what I saw: modfy.video - 258 seconds native ffmpeg - 23 seconds This was using a 2020 MBP with a 2.3 GHz Quad-Core Intel Core i7 processor & 32GB of RAM. My local ffmpeg version was v4.3.1. So it looks like performance definitely takes a hit due to WASM, but I'm sure performance will improve, and this is technically a very impressive achievement. The UI is really nice, and the privacy benefits of remaining client side, whilst delivered through the Web are great (i.e. nothing to install locally). I'm one of the co-founders @ Zamzar so really interested in this space. Congrats to @cryogenicplanet on this project - you've done a great job with it!
  - the-dude 6y ago
    Isn't .MOV already .MP4?
    - dylan604 6y ago
      No. MP4 is based on MOV. MP4 is typically going to have h.264 video with AAC audio. MOV is not limited in video codec, nor audio codec. Even though MediaInfo reports MOV as MPEG-4, they are not the same.
      - Denvercoder9 6y ago
        MP4 isn't limited in video codec or audio codec either, and it's quite common to have non-H.264 or non-AAC codecs in them. The only difference between MOV and MP4 are a few later created extensions, and the difference in every day use is negligble.
        dylan604 6y ago
        MP4 is an ISO standardized version of where MOV was at the time it was made a standard. MOV is an Apple product that they are free to update as they see fit. Apple has since modified the QT container regarding things like the moov atoms for aspect ratio, color systems (709 vs 609 etc), and all of the other similar fiddly bits. This is the major differences between the two to me.
ArtWomb 6y ago
Thanks for building, cryogenicplanet! Just wanted to add I really like the "no-upload" programming paradigm a lot. I use browser native webgl rendering internally for example to automate generation of static digital art assets. All samples and finished images are transferred via http. And its easy to run on any device: just fire up chrome and enter url. I'd like to do the same for the video asset generation pipeline as well. For which I do most ffmpeg processing on host. WASM build allows for browser based video generation on the fly ;)
- cryogenicplanet 6y ago
  Yeah this is definitely try to push that dream of not having to upload/download software and doing most if not all things on the browser itself
FraKtus 6y ago
I am not knowledgeable with WASM, can it enable threading in FFmpeg? That’s where FFmpeg shine...
- markdog12 6y ago
  Yes, SIMD as well, although they're not available on all browsers/devices.
leptons 6y ago
I'm curious about using this to transcode youtube videos in the browser. Wondering if CORS is going to be an issue. Looks like I found another rabbit hole to keep me occupied for a while.

kevincox 6y ago

This is a real cool tool and I would love to see more simple tools client-side only. However I have serious concerns about the performance. When developing a solver (https://ricochetrobots.kevincox.ca/ https://ricochetrobots.kevincox.ca/) I found that compared to the native performance running it in WASM was ~2x slower. (Running it on a phone compared to a laptop was another ~2x). However I doubt the native compiler was using large amounts of SIMD. IIUC video encoding often will use a large amount of SIMD instructions so I would expect the slowdown to be worse. On the other hand SIMD support in WASM is in progress: https://www.chromestatus.com/feature/6533147810332672 https://www.chromestatus.com/feature/6533147810332672 (preview available on chromium)

londons_explore 6y ago
I too would like to know if the performance shortcomings of webassembly are inherent to its design (ie. The sandboxing mechanism), or are just due to a poor implementation. Basic things like not being able to get stuff from CPU to screen without multiple memcpy's are all problematic too...
- oscargrouch 6y ago
  Not a compiler expert, but looking at the bytecode it looked fairly limited and minimal number of opcodes. Giving it has to be universal, i imagine where a LLVM bitcode op might translate to 1, 2 or 3 native opcodes, with WebAssembly you might with much more opcodes in the native output. I mean, of course they will be able to make it better with time and effort, but i dont think we can spect a much bigger speedup in comparison with the native equivalent. Note: this is all very speculative, but by looking at it, it looked great as an accelerator, specially for math stuff, but i imagine that it will be hard even to beat the Javascript code for generic stuff giving how much it was already optimized for it. If any compiler expert here that is familiar with it want to correct in my assumptions, please feel free to do it. (Not event touching in issues like the ones you mention, like memory copy, where the compiler cannot translate the thing in a more optmized manner giving it cannot speculate about the environment and optimize for it)
cryogenicplanet 6y ago

TorchAtlas 6y ago

This is very interesting! Would definitely be useful non-technical users. I have a few UX concerns, though. - By default, it's set to All Features instead of Basic Features - The app repeatedly says it doesn't upload your files, yet there's a big button in the app saying Upload File. Perhaps "Your files are never uploaded" should be reworded to something like "Your files never leave your device"? - It also was not obvious how to get started with the app. Perhaps some text could be added to the play button?

dylan604 6y ago
Or change the text on the button. 'Add Files', 'Import Files', etc would be equally useful while not confusing.
cryogenicplanet 6y ago
Thanks for the feedback, will definitely take this into account and iterate on this feedback. Someone opened an issue related to this too, so will update it on that issue https://github.com/Etwas-Builders/modfy.video/issues/88 https://github.com/Etwas-Builders/modfy.video/issues/88

jszymborski 6y ago

Wow! I wish I saw this a couple of months ago when I set up server-side transcoding for a client ;_;

deleted 6y ago

[deleted]

hardwaregeek 6y ago

I tried running something like this on my own, but I got OOM errors pretty quickly. 4 gigs isn't a lot when it comes to video processing. Hopefully wasm64 and WebGPU will make it a lot better.

cryogenicplanet 6y ago
I think this is as a result of the video have to be uploaded into RAM to be processed, this is definitely something I want to try to fix where I access the videos on the native file system to allow much larger files

cbhl 6y ago

On the left side, the label says "Click or Drag to upload", and at the bottom there is a label saying "Your files are not uploaded anywhere". If you have control over the left file picker, it might make sense to change the label to "Click or Drag to pick a video" or similar. Really great to see this -- I once worked on a cloud-based video editor and the cost of running ffmpeg in the cloud was surprisingly prohibitive.

cryogenicplanet 6y ago
Thanks! Already tracking the label issue here https://github.com/Etwas-Builders/modfy.video/issues/88 https://github.com/Etwas-Builders/modfy.video/issues/88 will resolve that tonight I think that's another great thing for me with this product, is that it almost costs nothing to run it (Thanks netlify)

alfg 6y ago

This is pretty cool. I've also been playing around in this area lately and noticed there's still a drastic performance hit. But seems to work great for smaller files. Question: Are you compiling FFmpeg to WASM, or FFMpeg's libav and interfacing with that directly? Also for anyone curious, the author of ffmpeg.wasm[1] has an excellent guide on compiling FFmpeg to WASM: https://itnext.io/build-ffmpeg-webassembly-version-ffmpeg-js-part-1-preparation-ed12bf4c8fac https://itnext.io/build-ffmpeg-webassembly-version-ffmpeg-js... [1] https://github.com/ffmpegwasm/ffmpeg.wasm https://github.com/ffmpegwasm/ffmpeg.wasm

ggonweb 6y ago

Is there a restriction on the size of the file or can it save the contents on the fly to disk like streamsaver?

cryogenicplanet 6y ago
Currently there is a restriction on size of file because it has to process the video in your memory and isn't saving content on the fly to disk but that is definitely something I can try to add moving forward
- ggonweb 6y ago
  Good job on your release, Curious, How is your version different from kagami or videoconverter versions, which were file conversions and they did not have streaming options, even though they had some patches who showed how to do it. Does yours support streaming options so that the streams can be saved to file serially using streamsaver.js.

Akaolisa 6y ago

Very nice