4 ms·

As a little side project, I've been trying to automate creation of those "1 second everyday" style videos [1], and used FFmpeg to achieve this. For things like

by umaar 6y ago

As a little side project, I've been trying to automate creation of those "1 second everyday" style videos [1], and used FFmpeg to achieve this. For things like trimming and concatenating videos, one thing that surprised me was that it was slower than using a tool like ScreenFlow. Note, we're talking about hundreds of gigabytes worth of 4K videos. slower = When I say slower, I mean, if I manually performed the same operation in a professional video editing tool like ScreenFlow, the time it took ScreenFlow to export a video was quicker than the time it took FFmpeg to finish executing the command. Interestingly, there seems to be a fast and a slow way to do things in FFmpeg [2]. The slow way is free of quirks, whereas the fast way introduces something unexpected to the video, like a half a second of a black screen with audio continue playing like normal. I'm still curious as to how a tool like ScreenFlow can achieve faster trimming/concatenation/subtitle overlaying, than FFmpeg. I suspect if I read their documentation and do some more research, I'll discover a more optimal way of ordering the various flags on the command line which can speed up the execution, while preserving accuracy. [1] https://github.com/umaar/video-everyday https://github.com/umaar/video-everyday [2] https://superuser.com/questions/499380/accurate-cutting-of-video-audio-with-ffmpeg https://superuser.com/questions/499380/accurate-cutting-of-v...

vanderZwan 6y ago
> "1 second everyday" style videos Tangent: you reminded me of one of the coolest auditory experiences I've ever had. Roughly one-and-a-half decades ago I attended a public lecture by Olivier Nijs, a sound design guy from my region[0], about how he built an automated set-up from an old desktop to record one second of 7:00 in the morning every day. Then he manually cut together one whole year. The amazing thing about it was that after a few seconds the long-term trends really started to become noticeable. The changing sounds of birds, people and other living things. How rains in spring were somehow just a little different than the rains in summer or autumn. It was really, really amazing. (the artist himself wasn't that impressed with his own work - perhaps he saw someone else do it before and didn't feel like showing off with something unoriginal or something?) [0] https://www.oliviernijs.nl/ https://www.oliviernijs.nl/
- nneonneo 6y ago
  Not being able to read Dutch, do you happen to have a direct link to the lecture or the recording he made?
  - Cactus2018 6y ago
    i'm thinking it's audio only. Maybe it's hiding here? https://soundcloud.com/oliviernijs https://soundcloud.com/oliviernijs
    - chewzerita 6y ago
      My guess is this one: https://soundcloud.com/oliviernijs/perdag1900 https://soundcloud.com/oliviernijs/perdag1900
      - 0xdeadb00f 6y ago
        Sounds about right
      - vanderZwan 6y ago
        That's seven in the afternoon, I guess I misremembered! Thank you so much for finding it! :) EDIT: This is six in the morning, maybe I heard that version https://soundcloud.com/oliviernijs/perdag600 https://soundcloud.com/oliviernijs/perdag600
- umaar 6y ago
  Sounds fascinating, stuff like that interests me a lot. Hope that video surfaces one day, would really like to see it.
- jancsika 6y ago
  Mm, that's one of those things I read and wish I had thought to try that. :)
ubercow13 6y ago
It depends what you're doing, there are many different ways to cut a section out of a video. If you do it copying the streams (not reencoding anything) ffmpeg should do it at almost the speed that it can read and write to disk. However if you are reencoding the video, the speed will depend on every parameter controlling the encoder. If that's what ScreenFlow is doing, it's probably using hardware accelerated encoding too.
Youden 6y ago
> The slow way is free of quirks, whereas the fast way introduces something unexpected to the video, like a half a second of a black screen with audio continue playing like normal. The reason here is the the "fast way" and the "slow way" work very differently behind the scenes. The "fast way" looks only needs to look at the container, the bits of metadata that tell a player which bits of data need to be given to the decoder at which time. It can just take a blob of data and stick it in another blob of data without looking at the contents. The "slow way" actually decodes the frames, that is, it takes the blobs of compressed data and turns them into actual pixels, which especially in the case of 4K video, is very slow. The reason the "fast way" might be less accurate is that the frame you're asking for might not be possible to obtain without decoding the video. Modern video codecs have different kinds of frames and some frames depend on the frames before or after them. If you took such a frame and just jammed it into another video, things would break, because the other frames it refers to are missing. > I'm still curious as to how a tool like ScreenFlow can achieve faster trimming/concatenation/subtitle overlaying, than FFmpeg. It's likely that they have optimisations that FFmpeg doesn't or cannot have. FFmpeg has a bit of an emphasis of being able to play and handle pretty much anything you throw at it, no matter how broken. It could be that accurate input seeking is difficult while preserving that reliability. One option that's probably not an option for you but you might consider is encoding your video in an all-intra format. This is fairly standard in the video editing world. All-intra means that all your frames are independent of each other and can be moved around by editing software without decoding anything. Doing this will result in larger files, however.
- umaar 6y ago
  This is a great explanation thank you! Makes much more sense now. Will indeed read into the all-intra format.
- scottlamb 6y ago
  Agree overall, but a couple nits: > The "slow way" actually decodes the frames, that is, it takes the blobs of compressed data and turns them into actual pixels, which especially in the case of 4K video, is very slow. The "slow way" decodes and re-encodes the frames. Decoding is a little slow. Encoding is very very slow if done at high quality. (And even high quality is still lossy.) > The reason the "fast way" might be less accurate is that the frame you're asking for might not be possible to obtain without decoding the video. This is often possible to solve with an .mp4 "edit list". You include more data than is expected to be displayed along with instructions for the player to skip part of it. One obvious caveat is that the person you send the video from can remove the edit list, so the hidden frames shouldn't be anything you want to redact for privacy.
g_airborne 6y ago
Most likely you are transcoding the video instead of copying the raw stream. A lot of more complicated stuff requires that but things like trimming can be done the fast way by simply cutting of the irrelevant pieces in the raw encoded data itself which is much faster. It’s kind of a sport to find the exact string of flags that has the correct effect without transcoding :) The reason it often fails is that ffmpeg can do so many things that any time you are using some curious combination of flag A, B and C is likely that no one else has ever done that and there are some side effects ;) Anyway, some of it can be avoided by learning how containers en codecs work, what I-frames are and all the other nitty gritty details of the world of video where there is so much to learn! Here’s a great intro to get started for anyone who got curious: https://github.com/leandromoreira/digital_video_introduction https://github.com/leandromoreira/digital_video_introduction
- cyborgx7 6y ago
  This is the biggest weakness of ffmpeg, in my opinion. It oftentimes requires so much understanding of how things actually work, and has too little to offer in the way of abstracting those things away.
- bambax 6y ago
  Yes, copying the raw stream is the way to go if you're not rescaling, etc. This is the command to extract 15 seconds of video starting at the first minute, and not reencoding; it should be quite fast and it's also quite self-explanatory: ffmpeg -ss 00:01:00 -i in.mp4 -t 00:00:15 -c copy out.mp4 I'm really no expert so I just keep a few of those commands around... You don't need a deep understanding of video streams, etc. just to use ffmpeg.
reassembled 6y ago
Did you configure FFMPEG to encode your video with the exact same encoder as ScreenFlow? If Screenflow is using hardware accelerated encoding and FFMPEG was configured for software x264 encoding, that might explain the discrepancy.

by umaar 6y ago

vanderZwan 6y ago

> "1 second everyday" style videos Tangent: you reminded me of one of the coolest auditory experiences I've ever had. Roughly one-and-a-half decades ago I attended a public lecture by Olivier Nijs, a sound design guy from my region[0], about how he built an automated set-up from an old desktop to record one second of 7:00 in the morning every day. Then he manually cut together one whole year. The amazing thing about it was that after a few seconds the long-term trends really started to become noticeable. The changing sounds of birds, people and other living things. How rains in spring were somehow just a little different than the rains in summer or autumn. It was really, really amazing. (the artist himself wasn't that impressed with his own work - perhaps he saw someone else do it before and didn't feel like showing off with something unoriginal or something?) [0] https://www.oliviernijs.nl/ https://www.oliviernijs.nl/

nneonneo 6y ago
Not being able to read Dutch, do you happen to have a direct link to the lecture or the recording he made?
- Cactus2018 6y ago
  i'm thinking it's audio only. Maybe it's hiding here? https://soundcloud.com/oliviernijs https://soundcloud.com/oliviernijs
  - chewzerita 6y ago
    My guess is this one: https://soundcloud.com/oliviernijs/perdag1900 https://soundcloud.com/oliviernijs/perdag1900
    - 0xdeadb00f 6y ago
      Sounds about right
    - vanderZwan 6y ago
      That's seven in the afternoon, I guess I misremembered! Thank you so much for finding it! :) EDIT: This is six in the morning, maybe I heard that version https://soundcloud.com/oliviernijs/perdag600 https://soundcloud.com/oliviernijs/perdag600
umaar 6y ago
Sounds fascinating, stuff like that interests me a lot. Hope that video surfaces one day, would really like to see it.
jancsika 6y ago
Mm, that's one of those things I read and wish I had thought to try that. :)

ubercow13 6y ago

It depends what you're doing, there are many different ways to cut a section out of a video. If you do it copying the streams (not reencoding anything) ffmpeg should do it at almost the speed that it can read and write to disk. However if you are reencoding the video, the speed will depend on every parameter controlling the encoder. If that's what ScreenFlow is doing, it's probably using hardware accelerated encoding too.

Youden 6y ago

> The slow way is free of quirks, whereas the fast way introduces something unexpected to the video, like a half a second of a black screen with audio continue playing like normal. The reason here is the the "fast way" and the "slow way" work very differently behind the scenes. The "fast way" looks only needs to look at the container, the bits of metadata that tell a player which bits of data need to be given to the decoder at which time. It can just take a blob of data and stick it in another blob of data without looking at the contents. The "slow way" actually decodes the frames, that is, it takes the blobs of compressed data and turns them into actual pixels, which especially in the case of 4K video, is very slow. The reason the "fast way" might be less accurate is that the frame you're asking for might not be possible to obtain without decoding the video. Modern video codecs have different kinds of frames and some frames depend on the frames before or after them. If you took such a frame and just jammed it into another video, things would break, because the other frames it refers to are missing. > I'm still curious as to how a tool like ScreenFlow can achieve faster trimming/concatenation/subtitle overlaying, than FFmpeg. It's likely that they have optimisations that FFmpeg doesn't or cannot have. FFmpeg has a bit of an emphasis of being able to play and handle pretty much anything you throw at it, no matter how broken. It could be that accurate input seeking is difficult while preserving that reliability. One option that's probably not an option for you but you might consider is encoding your video in an all-intra format. This is fairly standard in the video editing world. All-intra means that all your frames are independent of each other and can be moved around by editing software without decoding anything. Doing this will result in larger files, however.

umaar 6y ago
This is a great explanation thank you! Makes much more sense now. Will indeed read into the all-intra format.
scottlamb 6y ago
Agree overall, but a couple nits: > The "slow way" actually decodes the frames, that is, it takes the blobs of compressed data and turns them into actual pixels, which especially in the case of 4K video, is very slow. The "slow way" decodes and re-encodes the frames. Decoding is a little slow. Encoding is very very slow if done at high quality. (And even high quality is still lossy.) > The reason the "fast way" might be less accurate is that the frame you're asking for might not be possible to obtain without decoding the video. This is often possible to solve with an .mp4 "edit list". You include more data than is expected to be displayed along with instructions for the player to skip part of it. One obvious caveat is that the person you send the video from can remove the edit list, so the hidden frames shouldn't be anything you want to redact for privacy.

g_airborne 6y ago

Most likely you are transcoding the video instead of copying the raw stream. A lot of more complicated stuff requires that but things like trimming can be done the fast way by simply cutting of the irrelevant pieces in the raw encoded data itself which is much faster. It’s kind of a sport to find the exact string of flags that has the correct effect without transcoding :) The reason it often fails is that ffmpeg can do so many things that any time you are using some curious combination of flag A, B and C is likely that no one else has ever done that and there are some side effects ;) Anyway, some of it can be avoided by learning how containers en codecs work, what I-frames are and all the other nitty gritty details of the world of video where there is so much to learn! Here’s a great intro to get started for anyone who got curious: https://github.com/leandromoreira/digital_video_introduction https://github.com/leandromoreira/digital_video_introduction

cyborgx7 6y ago
This is the biggest weakness of ffmpeg, in my opinion. It oftentimes requires so much understanding of how things actually work, and has too little to offer in the way of abstracting those things away.
bambax 6y ago
Yes, copying the raw stream is the way to go if you're not rescaling, etc. This is the command to extract 15 seconds of video starting at the first minute, and not reencoding; it should be quite fast and it's also quite self-explanatory: ffmpeg -ss 00:01:00 -i in.mp4 -t 00:00:15 -c copy out.mp4 I'm really no expert so I just keep a few of those commands around... You don't need a deep understanding of video streams, etc. just to use ffmpeg.

reassembled 6y ago

Did you configure FFMPEG to encode your video with the exact same encoder as ScreenFlow? If Screenflow is using hardware accelerated encoding and FFMPEG was configured for software x264 encoding, that might explain the discrepancy.