7 ms·

Show HN: Open-source AI video editor

268 points by drochetti 2y ago

Hey HN community! I'm one of the lead devs of this project at fal.ai and we created an open source lightweight video editor powered by the latest media AI models. The main goal was to tackle some challenges when dealing with complex media handling and encoding on the browser. It all started as an internal experiment but as we tackled some of the issues it was clear there could be some value sharing it with the open source community. Some of the key points and tech stack details: - It uses IndexedDb, so all data is local (i.e. no auth, no cloud db) - Multiple AI models for video, image, music and voice-over. APIs are provided by fal.ai - Built with the typical React+Next.js, Shadcn front-end - Used remotion.dev for the realtime video preview (this is such a great project, without it the codebase would be twice as large) - File uploads so you can bring your own media by uploadthing.com - ffmpeg for encoding the final video and also some ui tricks, like the audio waveform We deployed a version of it and for now it's free to use. We do plan to add some rate limiting and a bring your own API Key next, but it's open source and I'm curious about what the community will build on top of it, or derive from it. Customize your own video app and if you do, please share. If you have any questions, hit me up!

aykutkardas 2y ago
[flagged]
- S0y 2y ago
  100% AI generated comment?
oldgradstudent 2y ago
Cool project, I played with it a bit a few hours ago. A small bug report: if you have both a still image and a video in the timeline, the video does not play in the preview.
- aykutkardas 2y ago
  Glad you liked it. And thanks for reporting, I'll look into the issue.
asadm 2y ago
this looks great! I would like standard timeline scrup, drag-drop etc. Some eta on how long a generation will likely take.
- drochetti 2y ago
  Thanks for the feedback, eta would be great indeed. I'll look into it.
- kfarr 2y ago
  Second on timeline scrub (move timeline position depending on x position of click on timeline) and drag and drop asset from media into timeline. Also spacebar is usually a basic universal shortcut to play/pause
  - drochetti 2y ago
    Drag and drop the media to the timeline, and drag the media along the timeline track is already supported. We will keep improving the UI, including shortcuts. Thanks a lot for the feedback.
voiper1 2y ago
Wow, that's so fun. Just upload and generate assets super easily... queue the next one while it's working.
peppertree 2y ago
Open source part is just a thin layer of UI. Built to peddle their closed backend, which is built on others' open source work. Nice.
- echelon 2y ago
  Fal wants to sell compute, and since they're straggling behind the compute-as-an-API leaders they've attached their cart to the art, image, and video subset of the market. They think they've got a leg up on their model partners. That the foundation model companies will mostly become generic copies of one another, servants of the compute layer. There are so many foundation video models now, and they'll battle it out over dwindling margins. Pika, Runway, Kling - they're all the same. And there's also growing open source foundation models. The thing that stands in Fal's way is that the future of AI video for artists is local. Hunyuan and Comfy can run on desktop machines, and real artists hate the SaaS model and non local stuff. It doesn't look like we'll even need A100s to reach Pixar levels of quality. The ones to watch in the art space are Comfy and Invoke. And Adobe. Fal probably has a future in powering the API-driven YouTube Shorts slop, though there's probably an upper limit to the number of talking head startups. But there's no way they win artists over to cloud. Sophisticated tools are going to be local. Any SaaS art tools that do achieve escape velocity will buy and manage their own compute and won't go through Fal as a reseller.
  - bobxmax 2y ago
    [flagged]
    - echelon 2y ago
      > This is just naive cope. Seems to me like you're not watching a growing trend. I'm deeply embedded in this community and comfy is the biggest thing in it. https://trends.google.com/trends/explore?date=today%205-y&q=midjourney,comfyui https://trends.google.com/trends/explore?date=today%205-y&q=... https://trends.google.com/trends/explore?date=today%205-y&q=runwayml,comfyui https://trends.google.com/trends/explore?date=today%205-y&q=... https://trends.google.com/trends/explore?date=today%205-y&q=pika%20labs,comfyui https://trends.google.com/trends/explore?date=today%205-y&q=... Everyone is tired of paying for generations they don't use. Everyone wants local. Local isn't some perfect little fisher price UI that some product manager approved. Local can be mixed and matched and pushed to the limits. Topaz Labs' run rate is more than Runway and Pika, so don't write off local tools.
mmh0000 2y ago
[flagged]
- deleted 2y ago
  [deleted]
lastdong 2y ago
Thank you for open sourcing the project, the UI looks fantastic. I noticed that several comments have mentioned the potential to improve functionality by allowing users to replace the backend with local models, such as Hunyuan. Would you be open to considering this feature in the future? Additionally, if other developers were to contribute, do you think this could be a possibility?
- drochetti 2y ago
  Absolutely. The idea of being open source with a permissive license is that we're encouraging anyone to do whatever fits their use case. You can replace anything, deploy on your own server, port it to other stacks... whatever brings value to you. We're also open to PRs, cut an issue in the repo and we can get the conversation going.
Jayakumark 2y ago
Great release , where is the source code for FFMPEG library used?
SrslyJosh 2y ago
[flagged]
- deleted 2y ago
  [deleted]
- bdangubic 2y ago
  if this comment was a video I’d edit it with this tool :)
- dang 2y ago
  Could you please stop posting unsubstantive comments and flamebait? Your account has unfortunately been doing this repeatedly, e.g.: https://news.ycombinator.com/item?id=42789451 https://news.ycombinator.com/item?id=42789451 https://news.ycombinator.com/item?id=42742537 https://news.ycombinator.com/item?id=42742537 https://news.ycombinator.com/item?id=42742525 https://news.ycombinator.com/item?id=42742525 That's not what this site is for, and destroys what it is for, so we end up having to ban such accounts. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
satvikpendem 2y ago
I was just looking for something like this, is this similar to RunwayML and their editor?
archerx 2y ago
[flagged]
- deleted 2y ago
  [deleted]
popalchemist 2y ago
Wow, very impressive. Is there any way to use the video editor UI component in a package, separately from the rest of this project?
- drochetti 2y ago
  Thanks! Not as a component, but you can clone or copy it and modify as needed. It's a quite complex UI, so it's not easy to export it as a single component.
- newusertoday 2y ago
  you can use remotion which this project is using
dylan604 2y ago
Am I missing a demo video or anything, or is the only way to see it to install and run it? The only thing I'm seeing is the poster image with what I'm assuming is a very cropped image of part of the UI??? Just seeing the interface gives a lot of clues as to potential usability as if it's targeted at novices or hardcore editors. Not being able to see if there's any kind of lag when trying to scrub/navigate the video is also something that tells me if I'm going to get upset with it from being web based or if it feels native. Just food for thought on what people might be looking for instead of having to install something that might not be right for them.
- thangngoc89 2y ago
  It’s here: https://fal-video-studio.vercel.app/ https://fal-video-studio.vercel.app/
  - dylan604 2y ago
    and you found that link how? am I just missing it in TFA? Edit: What do the buttons next to the play/pause button do? Typically, I'd expect to frame advance, or jump to next the next clip. I click and see nothing happening. I dropped two images I uploaded into the timeline, yet there's nothing visible in the timeline. The monitor shows something, but nothing in the timeline indicating how the clip fits in the timeline. There's not indication of what the frame size of the timeline is. The images look like they are being stretched/squeezed to fit, but there's also no indication in the media bin about the media like duration/size/etc. This really feels targeted for people with no previous knowledge of a video editor. It also just has that feel of "do something quick to make a post with it's AI to HN".
    - drochetti 2y ago
      The link is on the GitHub repo. As for the feedback, fair enough. As I said, this is just a demo and it's in its early stages. It has no goal of replacing professional video editors or even matching their features. Some annoying bugs and missing features, we will tackle them. As for your last comment, feel free to judge, but I can say it was not quick at all to do this just to post on HN. You can check all the commit history and the codebase so you can have an idea how much care was put into it.
nikolayasdf123 2y ago
someone please add "AI Video Editing". I don't care about generating assets with AI. scrolling frame by frame through hours of hundreds of tiny videos, fixing their exposure, fixing their dropped frames, fixing transitions, fixing and matching visual narration to textual and audio narration, aligning music to video, making "vlog" like templates. there is just so much to be fixed in this industry, but everyone is concerned with generating assets. that's just not the biggest issue video editors are facing. like not at all.
- EyMaddis 2y ago
  Because for generation you just have to make a prompt to OpenAI and get a VC to throw money at you. Why bother with the hard thing
  - nikolayasdf123 2y ago
    you don't even need VC. people would love it and more than happy to pay for it, high price! I bet China has already some of such apps. I see many very similar vlogs in Little Red Note, likely edited automatically by some kind of software. build it and. ship it
- t0bia_s 2y ago
  Absolutely agree. I don't care about generating footages with non existing content, just let me help with editing of already taken real life footages to speed up post-production!
- burningion 2y ago
  I'm working on this! Here's an example using Claude and Model Context Protocol: https://www.youtube.com/watch?v=bEDA9lkkGVM https://www.youtube.com/watch?v=bEDA9lkkGVM You can sign up at https://www.video-jungle.com/ https://www.video-jungle.com/ and I'll send an invite over.
  - farzd 2y ago
    very cool, been looking to make some applications using something like this. What kinda of pricing model is this going to have ?
dgreensp 2y ago
Thanks for posting it! Remotion does look really cool for in-browser video editing. People should note that you can’t make free-to-use software with Remotion. It is a paid product, and you can download and run the code for free in very limited circumstances that let you experiment with it. You can’t use the output of your own software built on Remotion for any for-profit purpose, nor can anyone else, without a license.
wanderingmind 2y ago
Please add a dockerfile to generate a docker image that makes it easy to setup and test
atentaten 2y ago
Nice work! It would be cool to be able to do a basic cross fade along with fade in fade out with the clips.
jerriclynsjohn 2y ago
Hey, co-founder of oss.incredible.dev here. @drochetti this is an amazing project, I love the codebase and I honestly think this is the future of video production. During the days of oss.incredible.dev we found that the biggest friction for our user in video production was themselves, It's extremely hard for anyone to come in front of a camera and record themselves, while we were able to solve for everything else like scripting, story boarding, layouts, animations, collaboration, block based recording etc it was the inability to record themselves that caused them to leave the workspace hanging. With AI, users won't have to record themselves, they can have a saved avatar and just give it a script to make it work! This was the missing puzzle. I'm planning to revive oss.incredible.dev, but with the intention to make it AI first product. Let me know if anyone here would like to join hands.
- grugagag 2y ago
  It is quite a terrible thing if AI technology completely replaces the human with their own flaws with something that mimics human but is completely devoid of it and obediently executes the deed, reads the scripts that is fed and never fails. Yes it is more efficient this way but is this consumption geared towards humans or just to the human as cattle? Seems perfect for propaganda though.
  - jerriclynsjohn 2y ago
    I think about this as democratization of content creation, now people who were shy but have immense knowledge can share what they know to the world. Think about the accessibility unlock that happens with a product like this. The biggest problem we saw when we created incredible.dev was that extremely talented developers were reluctant to come in front of a camera, but they wrote amazing scripts. They knew what they were talking about, but the only blocker was themselves. This product can unlock knowledge like never before, people get to stay relevant in the fast-paced video first world.
rebalh 2y ago
hey man cool thing, one advice, add a video on the GitHub page that shows how the thing works.
steve_aldrin 2y ago
can we add smooth zoom in effects to particular part of vids? like the ones used in product demos etc
yigitkonur35 2y ago
great stuff! nice to see that remotion is becoming more popular on such projects.

268 points by drochetti 2y ago

aykutkardas 2y ago

[flagged]

S0y 2y ago
100% AI generated comment?

oldgradstudent 2y ago

Cool project, I played with it a bit a few hours ago. A small bug report: if you have both a still image and a video in the timeline, the video does not play in the preview.

aykutkardas 2y ago
Glad you liked it. And thanks for reporting, I'll look into the issue.

asadm 2y ago

this looks great! I would like standard timeline scrup, drag-drop etc. Some eta on how long a generation will likely take.

drochetti 2y ago
Thanks for the feedback, eta would be great indeed. I'll look into it.
kfarr 2y ago
Second on timeline scrub (move timeline position depending on x position of click on timeline) and drag and drop asset from media into timeline. Also spacebar is usually a basic universal shortcut to play/pause
- drochetti 2y ago
  Drag and drop the media to the timeline, and drag the media along the timeline track is already supported. We will keep improving the UI, including shortcuts. Thanks a lot for the feedback.

voiper1 2y ago

Wow, that's so fun. Just upload and generate assets super easily... queue the next one while it's working.

peppertree 2y ago

Open source part is just a thin layer of UI. Built to peddle their closed backend, which is built on others' open source work. Nice.

echelon 2y ago
Fal wants to sell compute, and since they're straggling behind the compute-as-an-API leaders they've attached their cart to the art, image, and video subset of the market. They think they've got a leg up on their model partners. That the foundation model companies will mostly become generic copies of one another, servants of the compute layer. There are so many foundation video models now, and they'll battle it out over dwindling margins. Pika, Runway, Kling - they're all the same. And there's also growing open source foundation models. The thing that stands in Fal's way is that the future of AI video for artists is local. Hunyuan and Comfy can run on desktop machines, and real artists hate the SaaS model and non local stuff. It doesn't look like we'll even need A100s to reach Pixar levels of quality. The ones to watch in the art space are Comfy and Invoke. And Adobe. Fal probably has a future in powering the API-driven YouTube Shorts slop, though there's probably an upper limit to the number of talking head startups. But there's no way they win artists over to cloud. Sophisticated tools are going to be local. Any SaaS art tools that do achieve escape velocity will buy and manage their own compute and won't go through Fal as a reseller.
- bobxmax 2y ago
  [flagged]
  - echelon 2y ago
    > This is just naive cope. Seems to me like you're not watching a growing trend. I'm deeply embedded in this community and comfy is the biggest thing in it. https://trends.google.com/trends/explore?date=today%205-y&q=midjourney,comfyui https://trends.google.com/trends/explore?date=today%205-y&q=... https://trends.google.com/trends/explore?date=today%205-y&q=runwayml,comfyui https://trends.google.com/trends/explore?date=today%205-y&q=... https://trends.google.com/trends/explore?date=today%205-y&q=pika%20labs,comfyui https://trends.google.com/trends/explore?date=today%205-y&q=... Everyone is tired of paying for generations they don't use. Everyone wants local. Local isn't some perfect little fisher price UI that some product manager approved. Local can be mixed and matched and pushed to the limits. Topaz Labs' run rate is more than Runway and Pika, so don't write off local tools.

mmh0000 2y ago

[flagged]

deleted 2y ago
[deleted]

lastdong 2y ago

Thank you for open sourcing the project, the UI looks fantastic. I noticed that several comments have mentioned the potential to improve functionality by allowing users to replace the backend with local models, such as Hunyuan. Would you be open to considering this feature in the future? Additionally, if other developers were to contribute, do you think this could be a possibility?

drochetti 2y ago
Absolutely. The idea of being open source with a permissive license is that we're encouraging anyone to do whatever fits their use case. You can replace anything, deploy on your own server, port it to other stacks... whatever brings value to you. We're also open to PRs, cut an issue in the repo and we can get the conversation going.

Jayakumark 2y ago

Great release , where is the source code for FFMPEG library used?

SrslyJosh 2y ago

[flagged]

deleted 2y ago
[deleted]
bdangubic 2y ago
if this comment was a video I’d edit it with this tool :)
dang 2y ago
Could you please stop posting unsubstantive comments and flamebait? Your account has unfortunately been doing this repeatedly, e.g.: https://news.ycombinator.com/item?id=42789451 https://news.ycombinator.com/item?id=42789451 https://news.ycombinator.com/item?id=42742537 https://news.ycombinator.com/item?id=42742537 https://news.ycombinator.com/item?id=42742525 https://news.ycombinator.com/item?id=42742525 That's not what this site is for, and destroys what it is for, so we end up having to ban such accounts. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

satvikpendem 2y ago

I was just looking for something like this, is this similar to RunwayML and their editor?

archerx 2y ago

[flagged]

deleted 2y ago
[deleted]

popalchemist 2y ago

Wow, very impressive. Is there any way to use the video editor UI component in a package, separately from the rest of this project?

drochetti 2y ago
Thanks! Not as a component, but you can clone or copy it and modify as needed. It's a quite complex UI, so it's not easy to export it as a single component.
newusertoday 2y ago
you can use remotion which this project is using

dylan604 2y ago

Am I missing a demo video or anything, or is the only way to see it to install and run it? The only thing I'm seeing is the poster image with what I'm assuming is a very cropped image of part of the UI??? Just seeing the interface gives a lot of clues as to potential usability as if it's targeted at novices or hardcore editors. Not being able to see if there's any kind of lag when trying to scrub/navigate the video is also something that tells me if I'm going to get upset with it from being web based or if it feels native. Just food for thought on what people might be looking for instead of having to install something that might not be right for them.

thangngoc89 2y ago
It’s here: https://fal-video-studio.vercel.app/ https://fal-video-studio.vercel.app/
- dylan604 2y ago
  and you found that link how? am I just missing it in TFA? Edit: What do the buttons next to the play/pause button do? Typically, I'd expect to frame advance, or jump to next the next clip. I click and see nothing happening. I dropped two images I uploaded into the timeline, yet there's nothing visible in the timeline. The monitor shows something, but nothing in the timeline indicating how the clip fits in the timeline. There's not indication of what the frame size of the timeline is. The images look like they are being stretched/squeezed to fit, but there's also no indication in the media bin about the media like duration/size/etc. This really feels targeted for people with no previous knowledge of a video editor. It also just has that feel of "do something quick to make a post with it's AI to HN".
  - drochetti 2y ago
    The link is on the GitHub repo. As for the feedback, fair enough. As I said, this is just a demo and it's in its early stages. It has no goal of replacing professional video editors or even matching their features. Some annoying bugs and missing features, we will tackle them. As for your last comment, feel free to judge, but I can say it was not quick at all to do this just to post on HN. You can check all the commit history and the codebase so you can have an idea how much care was put into it.

nikolayasdf123 2y ago

someone please add "AI Video Editing". I don't care about generating assets with AI. scrolling frame by frame through hours of hundreds of tiny videos, fixing their exposure, fixing their dropped frames, fixing transitions, fixing and matching visual narration to textual and audio narration, aligning music to video, making "vlog" like templates. there is just so much to be fixed in this industry, but everyone is concerned with generating assets. that's just not the biggest issue video editors are facing. like not at all.

EyMaddis 2y ago
Because for generation you just have to make a prompt to OpenAI and get a VC to throw money at you. Why bother with the hard thing
- nikolayasdf123 2y ago
  you don't even need VC. people would love it and more than happy to pay for it, high price! I bet China has already some of such apps. I see many very similar vlogs in Little Red Note, likely edited automatically by some kind of software. build it and. ship it
t0bia_s 2y ago
Absolutely agree. I don't care about generating footages with non existing content, just let me help with editing of already taken real life footages to speed up post-production!
burningion 2y ago
I'm working on this! Here's an example using Claude and Model Context Protocol: https://www.youtube.com/watch?v=bEDA9lkkGVM https://www.youtube.com/watch?v=bEDA9lkkGVM You can sign up at https://www.video-jungle.com/ https://www.video-jungle.com/ and I'll send an invite over.
- farzd 2y ago
  very cool, been looking to make some applications using something like this. What kinda of pricing model is this going to have ?

dgreensp 2y ago

Thanks for posting it! Remotion does look really cool for in-browser video editing. People should note that you can’t make free-to-use software with Remotion. It is a paid product, and you can download and run the code for free in very limited circumstances that let you experiment with it. You can’t use the output of your own software built on Remotion for any for-profit purpose, nor can anyone else, without a license.

wanderingmind 2y ago

Please add a dockerfile to generate a docker image that makes it easy to setup and test

atentaten 2y ago

Nice work! It would be cool to be able to do a basic cross fade along with fade in fade out with the clips.

jerriclynsjohn 2y ago

Hey, co-founder of oss.incredible.dev here. @drochetti this is an amazing project, I love the codebase and I honestly think this is the future of video production. During the days of oss.incredible.dev we found that the biggest friction for our user in video production was themselves, It's extremely hard for anyone to come in front of a camera and record themselves, while we were able to solve for everything else like scripting, story boarding, layouts, animations, collaboration, block based recording etc it was the inability to record themselves that caused them to leave the workspace hanging. With AI, users won't have to record themselves, they can have a saved avatar and just give it a script to make it work! This was the missing puzzle. I'm planning to revive oss.incredible.dev, but with the intention to make it AI first product. Let me know if anyone here would like to join hands.

grugagag 2y ago
It is quite a terrible thing if AI technology completely replaces the human with their own flaws with something that mimics human but is completely devoid of it and obediently executes the deed, reads the scripts that is fed and never fails. Yes it is more efficient this way but is this consumption geared towards humans or just to the human as cattle? Seems perfect for propaganda though.
- jerriclynsjohn 2y ago
  I think about this as democratization of content creation, now people who were shy but have immense knowledge can share what they know to the world. Think about the accessibility unlock that happens with a product like this. The biggest problem we saw when we created incredible.dev was that extremely talented developers were reluctant to come in front of a camera, but they wrote amazing scripts. They knew what they were talking about, but the only blocker was themselves. This product can unlock knowledge like never before, people get to stay relevant in the fast-paced video first world.

rebalh 2y ago

hey man cool thing, one advice, add a video on the GitHub page that shows how the thing works.

steve_aldrin 2y ago

can we add smooth zoom in effects to particular part of vids? like the ones used in product demos etc

yigitkonur35 2y ago

great stuff! nice to see that remotion is becoming more popular on such projects.