3 ms·

Very cool tool, and always amazing what Emacs can do... If this isn’t your _daily_ use case and you only need to edit video from time to time, just ask your pr

Very cool tool, and always amazing what Emacs can do... If this isn’t your _daily_ use case and you only need to edit video from time to time, just ask your preferred LLM to give you the FFmpeg commands to cut, speed up, mute, flip, add text, etc. This has worked quite well for me with simple use cases, and well no need to learn Emacs.

latexr 1y ago
That’d be an awful way to cut video, because it wouldn’t help with the most important part: visualising and extracting the exact initial and final time stamps. Especially if you don’t do this regularly, a GUI makes much more sense and saves you the frustration of having to sift through potentially wrong commands and figuring out what exactly to edit to fix the mistakes.
- beklein 1y ago
  I’m not saying this is better than a GUI, but instead let me give you an example. Sometimes I have movie clips A, B, and C. I want to trim part of A at the beginning and end, then stitch clip B to it while speeding it up by 2x, and finally add clip C at the end, also trimmed a bit. After that, I want to add some text at specific times for a specific duration. In the end, I’ll export everything in 1080p. I know all of this can be done with Final Cut or any other video editing app using a GUI, or the cool Emacs tool above... But for me this would mean (GUI example) downloading the app and watching YouTube tutorials just to learn what to click. For simple video editing (and other tasks), I sometimes need to get the job done quickly without wanting to learn a whole new tool. I found out that I can achieve sufficiently advanced video edits with FFmpeg commands produced step by step from a LLM. If you think this is awful, ok. I thought it was neat and wanted to share the idea.
  - latexr 1y ago
    I’m having a hard time believing your example. If you ever needed to do that more than once, using a basic video editor (of which there are many, free and open-source, no need for a commercial behemoth like Final Cut), playing with it for ten minutes once would give you all the knowledge you need forever, even when you are without access to your LLM. And you can keep the app installed, you don’t need to download it every time. There’s also no need to watch YouTube videos, most of these basic editors have evident interfaces that anyone could figure out on their own for simple tasks. People did figure out things before YouTube tutorials. Or hey, if you’re that keen on LLMs, ask them where the option you want is. Furthermore, you have not addressed at all the crux of the point. How are you even getting the exact time stamps to give to the LLM of FFmpeg for the cut? Or how do you decide that 2x is the exact speedup you need? Or how do you know what size and position and text font and colour even make sense? All of those are visual decisions which need confirmation because video is visual. It doesn’t make sense to blindly run lengthy FFmpeg commands over and over to see if the result is any good.
    - beklein 1y ago
      Again, totally fair points, and for many people a (simple) GUI is the right tool. I am not against GUIs or particularly pro LLMs; I just want to show an alternative way to solve video editing problems without judging any specifc technology. Not all video work is visual-first storytelling. In engineering/lab contexts you often just need “good enough” trims, concatenation, speedups, and a few labels to document an experiment. As I said in another comment, I usually get the timestamps by noting them down while watching the video, or in rarer cases from timestamped sensor data. Sorry if my explanation wasn't good enough...
beepbooptheory 1y ago
If the LLM is not seeing the video, what does this add for you over just looking up the relevant options?
- StableAlkyne 1y ago
  > just looking up the relevant options Not the OP, but ffmpeg's documentation is pretty awful to navigate if you aren't an expert in it. It is very technical and exact, but fails to provide examples for most options. For example, the fpsmax option says you can set the maximum frame rate using "Hz value, fraction or abbreviation." So now the user has to search for how any of these things are to be specified (does a bare 60 mean 60hz? Do you need to include Hz? Is it case sensitive? What are the abbreviations? What is the fraction a fraction of?). And that's a random option I found in the pile of documentation it has. Maybe it's useful if you use it every day, but I completely understand why people use GUIs whose purpose is to construct the command. Or even LLMs to do the same thing.
  - shadowgovt 1y ago
    I strongly suspect this is the niche that LLMs will settle into long-term in programming work. They're pretty good at munging over code, documentation, and discourse online to paper over the issues with some projects having docs written in human-communication-as-a-second-language form, which is a lot of projects. We've already migrated (via StackOverflow) far past 'RTFM' as the go-to answer, and LLMs seem the next logical step there: a machine to read all the sources and boil them down for you.
- beklein 1y ago
  In my common use case, I note down the timestamps of the cuts. No LLM needed here. I know there are options for everything in FFmpeg, and I’m thankful to the community and maintainers for providing such a powerful and well-documented tool. I sometimes just want a quick video edit that doesn’t involve reading the man pages for minutes or hours.

latexr 1y ago

That’d be an awful way to cut video, because it wouldn’t help with the most important part: visualising and extracting the exact initial and final time stamps. Especially if you don’t do this regularly, a GUI makes much more sense and saves you the frustration of having to sift through potentially wrong commands and figuring out what exactly to edit to fix the mistakes.

beklein 1y ago
I’m not saying this is better than a GUI, but instead let me give you an example. Sometimes I have movie clips A, B, and C. I want to trim part of A at the beginning and end, then stitch clip B to it while speeding it up by 2x, and finally add clip C at the end, also trimmed a bit. After that, I want to add some text at specific times for a specific duration. In the end, I’ll export everything in 1080p. I know all of this can be done with Final Cut or any other video editing app using a GUI, or the cool Emacs tool above... But for me this would mean (GUI example) downloading the app and watching YouTube tutorials just to learn what to click. For simple video editing (and other tasks), I sometimes need to get the job done quickly without wanting to learn a whole new tool. I found out that I can achieve sufficiently advanced video edits with FFmpeg commands produced step by step from a LLM. If you think this is awful, ok. I thought it was neat and wanted to share the idea.
- latexr 1y ago
  I’m having a hard time believing your example. If you ever needed to do that more than once, using a basic video editor (of which there are many, free and open-source, no need for a commercial behemoth like Final Cut), playing with it for ten minutes once would give you all the knowledge you need forever, even when you are without access to your LLM. And you can keep the app installed, you don’t need to download it every time. There’s also no need to watch YouTube videos, most of these basic editors have evident interfaces that anyone could figure out on their own for simple tasks. People did figure out things before YouTube tutorials. Or hey, if you’re that keen on LLMs, ask them where the option you want is. Furthermore, you have not addressed at all the crux of the point. How are you even getting the exact time stamps to give to the LLM of FFmpeg for the cut? Or how do you decide that 2x is the exact speedup you need? Or how do you know what size and position and text font and colour even make sense? All of those are visual decisions which need confirmation because video is visual. It doesn’t make sense to blindly run lengthy FFmpeg commands over and over to see if the result is any good.
  - beklein 1y ago
    Again, totally fair points, and for many people a (simple) GUI is the right tool. I am not against GUIs or particularly pro LLMs; I just want to show an alternative way to solve video editing problems without judging any specifc technology. Not all video work is visual-first storytelling. In engineering/lab contexts you often just need “good enough” trims, concatenation, speedups, and a few labels to document an experiment. As I said in another comment, I usually get the timestamps by noting them down while watching the video, or in rarer cases from timestamped sensor data. Sorry if my explanation wasn't good enough...

beepbooptheory 1y ago

If the LLM is not seeing the video, what does this add for you over just looking up the relevant options?

StableAlkyne 1y ago
> just looking up the relevant options Not the OP, but ffmpeg's documentation is pretty awful to navigate if you aren't an expert in it. It is very technical and exact, but fails to provide examples for most options. For example, the fpsmax option says you can set the maximum frame rate using "Hz value, fraction or abbreviation." So now the user has to search for how any of these things are to be specified (does a bare 60 mean 60hz? Do you need to include Hz? Is it case sensitive? What are the abbreviations? What is the fraction a fraction of?). And that's a random option I found in the pile of documentation it has. Maybe it's useful if you use it every day, but I completely understand why people use GUIs whose purpose is to construct the command. Or even LLMs to do the same thing.
- shadowgovt 1y ago
  I strongly suspect this is the niche that LLMs will settle into long-term in programming work. They're pretty good at munging over code, documentation, and discourse online to paper over the issues with some projects having docs written in human-communication-as-a-second-language form, which is a lot of projects. We've already migrated (via StackOverflow) far past 'RTFM' as the go-to answer, and LLMs seem the next logical step there: a machine to read all the sources and boil them down for you.
beklein 1y ago
In my common use case, I note down the timestamps of the cuts. No LLM needed here. I know there are options for everything in FFmpeg, and I’m thankful to the community and maintainers for providing such a powerful and well-documented tool. I sometimes just want a quick video edit that doesn’t involve reading the man pages for minutes or hours.