14 ms·

FFmpeg 6.0

594 points by gyan 4y ago

gyan 4y ago
Changelog: * Radiance HDR image support * ddagrab (Desktop Duplication) video capture filter * ffmpeg -shortest_buf_duration option * ffmpeg now requires threading to be built * ffmpeg now runs every muxer in a separate thread * Add new mode to cropdetect filter to detect crop-area based on motion vectors and edges * VAAPI decoding and encoding for 10/12bit 422, 10/12bit 444 HEVC and VP9 * WBMP (Wireless Application Protocol Bitmap) image format * a3dscope filter * bonk decoder and demuxer * Micronas SC-4 audio decoder * LAF demuxer * APAC decoder and demuxer * Media 100i decoders * DTS to PTS reorder bsf * ViewQuest VQC decoder * backgroundkey filter * nvenc AV1 encoding support * MediaCodec decoder via NDKMediaCodec * MediaCodec encoder * oneVPL support for QSV * QSV AV1 encoder * QSV decoding and encoding for 10/12bit 422, 10/12bit 444 HEVC and VP9 * showcwt multimedia filter * corr video filter * adrc audio filter * afdelaysrc audio filter * WADY DPCM decoder and demuxer * CBD2 DPCM decoder * ssim360 video filter * ffmpeg CLI new options: -stats_enc_pre[_fmt], -stats_enc_post[_fmt], * -stats_mux_pre[_fmt] * hstack_vaapi, vstack_vaapi and xstack_vaapi filters * XMD ADPCM decoder and demuxer * media100 to mjpegb bsf * ffmpeg CLI new option: -fix_sub_duration_heartbeat * WavArc decoder and demuxer * CrystalHD decoders deprecated * SDNS demuxer * RKA decoder and demuxer * filtergraph syntax in ffmpeg CLI now supports passing file contents as option values, by prefixing option name with '/' * hstack_qsv, vstack_qsv and xstack_qsv filters
- exhilaration 4y ago
  This mostly looks like they're adding support for new codecs, there's only 4 new options in there, which seems rather paltry for a dot zero version release. Should we consider ffmpeg feature complete?
  - gyan 4y ago
    Dot zero releases basically signify major bumps of the core libs, and are not really synced with user-facing feature changes.
  - cm2187 4y ago
    I am still using 4.2. Does it make any difference if all I care about is h264 and hevc?
    - LtdJorge 4y ago
      More modern x264/x265 I guess
    - bsenftner 4y ago
      At a prior job we pinned to 4.2.3 because after that version an internal change massively impacted high frame rates. With a 1080p mp4 file on disk, and that disc being an ssd, version 4.2.3 delivers 700-800 frames per second, while all the versions after that 4.2.3 release - no other app or library changes - that same file may reach 300 fps. I spent some time trying to find the cause of that 50% cut in efficiency, but never found enough time. Curious to try this version.
      - esperent 4y ago
        > I spent some time trying to find the cause of that 50% cut in efficiency Presumably they have an issue tracker. Did you report this?
        bsenftner 4y ago
        When I looked, it had been reported already.
        cstejerean 4y ago
        do you have a link?
      - mixmastamyk 4y ago
        You are referring to (re)encoding a file?
        bsenftner 4y ago
- FranOntanaya 4y ago
  Nostalgia moment with the CrystalHD decoder deprecation. That came along the Intel Atom netbooks, one of the few ways we had to bump a bit their austere performance.
QuarterRoy 4y ago
I love software that just does what it’s supposed to do and is so good at it that it’s an underpinning for other projects. FFmpeg, curl, nmap, et., al.
- Aperocky 4y ago
  It is such a swiss knife kind of tool, there's just almost nothing it can't do when it comes to video, audio and image modifications. I wonder how many big companies uses it "in secret" that we don't know off, or distributes it with mentions buried deep underneath usage agreement.
  - shmerl 4y ago
    Google must be using it for Youtube. Wikimedia is using it as well.
    - jzombie 4y ago
      I believe it is embedded in Chromium as well.
      - shmerl 4y ago
        Firefox bundles it as well I think (at least Linux version).
        padenot 4y ago
        Firefox vendors a subset of ffmpeg, to decode royalty free codecs such as vp8/9, mp3, flac, on all OSes. We're using our own decoders/demuxers for other royalty free codecs for historical reasons. It can also use the system's ffmpeg to decode patented formats such as h264 or AAC, on desktop Linux, if the copy of ffmpeg present on the host includes non-free codecs (same on other OSes, with their respective system libraries).
    - komali2 4y ago
      They do, they say so here: https://support.google.com/youtube/answer/1722171#zippy=%2Ccontainer-mp%2Caudio-codec-aac-lc%2Cvideo-codec-h%2Cframe-rate%2Cbitrate%2Cvideo-resolution-and-aspect-ratio%2Ccolor-space https://support.google.com/youtube/answer/1722171#zippy=%2Cc... > YouTube does not recommend the RGB color matrix on uploads. In this case, YouTube initially sets the color matrix to unspecified before the standardization. It will then infer the color matrix using the color primaries during standardization. Note that sRGB TRC will convert to BT.709 TRC. YouTube re-tags the color primaries/matrix/TRC to BT.709 when it is not supported by FFmpeg colorspace conversion filter. Since FFmpeg is gpl2.1 I thought they had to make it easier to know they're using this, like under a "licenses" section, but I don't see anything under studio.youtube.com indicating this.
asicsp 4y ago
Changelog: https://github.com/FFmpeg/FFmpeg/blob/master/Changelog https://github.com/FFmpeg/FFmpeg/blob/master/Changelog
shmerl 4y ago
What's the story with VAAPI AV1 encoding? Does it exist anywhere or not yet?
- mayli 4y ago
  - VAAPI decoding and encoding for 10/12bit 422, 10/12bit 444 HEVC and VP9
  - shmerl 4y ago
    That's been there for a while. AV1 decoding also exists. But I'm curious what's going on with AV1 encoding.
- stonogo 4y ago
  It exists only as a patch from Intel (https://github.com/intel/cartwheel-ffmpeg/blob/master/patches/0071-lavc-vaapi-support-av1-encode.patch https://github.com/intel/cartwheel-ffmpeg/blob/master/patche...) but I haven't seen any efforts to upstream it yet.
  - shmerl 4y ago
    Thanks! At least changes look recent so there must be some plan to upstream it.
lxe 4y ago
Fun fact of the day: ffmpeg is written by Fabrice Bellard, who among other impressive things, wrote the JS PC emulator capable of running windows 2000 in a browser over 10 years ago that got me fascinated with emulators in general. https://bellard.org/jslinux/tech.html https://bellard.org/jslinux/tech.html
- galleywest200 4y ago
  He also wrote QEMU, which is probably far more impressive.
  - digianarchist 4y ago
    https://smartbear.com/blog/fabrice-bellard-portrait-of-a-super-productive-pro/ https://smartbear.com/blog/fabrice-bellard-portrait-of-a-sup...
    - MuffinFlavored 4y ago
      This is missing QuickJS :) https://bellard.org/ https://bellard.org/
  - mobilio 4y ago
    Also LZEXE back in time...
  - xwolfi 4y ago
    Sorry, windows 98 running in JS is way way WAY more impressive. It's just insane.
  - mhh__ 4y ago
    The most impressive thing is the LTE modem
- Jasper_ 4y ago
  Fabrice has not been involved in ffmpeg for well over 15 years now.
arjvik 4y ago
I absolutely love ffmpeg, but for the life of me I cannot understand how its pipeline system works. Each time I need to use it, I attempt to construct the command myself, but end up giving up and consulting StackOverflow. Amazingly, someone has usually done the exact thing I need to do and posted their command line to StackOverflow, so I'm never out of luck! How do I actually start understanding how ffmpeg works? I want to be an ffmpeg power user.
- reidjs 4y ago
  I’ve never had to directly use ffmpeg. What do you do that makes it so useful to you? Something in video production?
  - pmontra 4y ago
    Not parent but in my case yes, video production: ffmpeg to extract audio from captured video, process audio with Audacity, ffmpeg to cut video and merge it with the audio from Audacity.
    - smcnally 4y ago
      Also to quickly convert video formats and containers, scale and crop renders, alter speed, make gifs
- mmsimanga 4y ago
  This. And imagemagick.
  - Macha 4y ago
    Though I have used ffmpeg on images, too.
- Aperocky 4y ago
  The ordering of the arguments changes behavior, I don't think that's a good thing or desired outcome, at most it's a bug_feature. It tend to happen when big project rolls out batteries itself, in this case command line parsing.
scottlamb 4y ago
From the changelog [1]: > - DTS to PTS reorder bsf Interesting, I wonder what this is / why you'd want it. In particular, when you have the DTS but not the PTS. The recent gstreamer 1.22 release [2] had what I read as the opposite—calculate a plausible DTS from the order and PTS. They did a nice job of explaining why it's useful. AFAICT, this approach is the only viable way to get B frames to work properly from a received RTP stream. > H.264/H.265 timestamp correction elements ... Muxers are often picky and need proper PTS/DTS timestamps set on the input buffers, but that can be a problem if the encoded input media stream comes from a source that doesn't provide proper signalling of DTS, such as is often the case for RTP, RTSP and WebRTC streams or Matroska container files. Theoretically parsers should be able to fix this up, but it would probably require fairly invasive changes in the parsers, so two new elements h264timestamper and h265timestamper bridge the gap in the meantime and can reconstruct missing PTS/DTS. Looks like the ffmpeg thing is dts2pts_bsf.c. [3] I haven't really read the implementation, but I was hoping the comment at the top would illuminate things, but "Derive PTS by reordering DTS from supported streams" isn't enough for me. [1] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/release/6.0:/Changelog https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/rel... [2] https://gstreamer.freedesktop.org/releases/1.22/ https://gstreamer.freedesktop.org/releases/1.22/ [3] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/release/6.0:/libavcodec/dts2pts_bsf.c https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/rel...
- jbk 4y ago
  This bsf was done to fix the following ticket https://trac.ffmpeg.org/ticket/502 https://trac.ffmpeg.org/ticket/502
  - scottlamb 4y ago
    Thanks. Looks like the input in question was an .avi file. That's a container format I don't know anything about (and don't particularly care to). I suspect then it's not relevant for anything more modern.
    - brigade 4y ago
FrostKiwi 4y ago
> nvenc AV1 encoding support > QSV AV1 encoder That is captial 'A' A W E S O M E. FFmpeg is always on the bleeding edge, love it.
jbk 4y ago
If you want to know a bit more about what is important in this major release, you can watch my presentation about FFmpeg 6.0 at Fosdem: https://fosdem.org/2023/schedule/event/om_vlc/ https://fosdem.org/2023/schedule/event/om_vlc/ (Video + Slides) It’s a bit less dry than the changelog, notably for the evolutions of the APIs. What’s also important is the changes about the release schedule that we’ve been pushing with the community. Major version every year at the beginning of the year, with ABI and API break, minor releases during the year and an LTS every other year…
- AndyKelley 4y ago
  Thank you for doing this talk. I agree with you, news about ffmpeg progress should be more widespread. And by the way, I love your hat!
- vhiremath4 4y ago
  Gracias!
- mikae1 4y ago
  At 2m15s you even mention HN. :)
  - qwerty456127 4y ago
    This is very nice. I've been surprised how many people don't know HN, it can definitely use some promotion.
    - _0ffh 4y ago
      Please, no! We'd need way more dangs than are available.
      - adrian_mrd 4y ago
        From the Oxford Dictionary of Modern Slang, paperback edition from 2010 on page 65 "Dang: Expressing annoyance or antipathy, or as an intensive" ;)
ww520 4y ago
Isn’t there a language for configuring ffmpeg to make it do different things?
- mixmastamyk 4y ago
  The ffmpeg-python wrapper module is sorta like that.
ggm 4y ago
ffmpeg is my go-to re-encoder. I didn't know ffprobe existed until I read some "howto" and its also increadibly useful as a way to get fundamental video and audio stream data out as e.g. csv or json. So I can run an ffprobe script to get x,y info out, decide if the video needs re-encoding, pass to an ffmpeg call which does fast or veryfast settings to reset the x/y scale (for instance) It's also unquestionably 'self documenting' because all of the sheharazade 1001 options are listed in --help. The problem is knowing which one will make the horse speak.
- yboris 4y ago
  I've been so frustrated that FFprobe functionality is not part of FFmpeg. My app extracts screenshots from videos to create a beautiful gallery of videos. But even though I include FFmpeg already, I need a 50mb FFprobe executable to be bundled with my app just so that I can determine the width, height, duration, and fps of a video file! What is it that FFprobe does that FFmpeg couldn't do with a few extra pieces of exposed API? https://videohubapp.com/ https://videohubapp.com/ - https://github.com/whyboris/Video-Hub-App https://github.com/whyboris/Video-Hub-App https://github.com/whyboris/Video-Hub-App/blob/772b25bbd4b41be6b081e6fcb51e5babae9ae2e2/node/main-support.ts#L458 https://github.com/whyboris/Video-Hub-App/blob/772b25bbd4b41...
  - jasomill 4y ago
    ffmpeg and ffprobe are built against the exact same set of libraries, so they could certainly be combined into a single program if the ffmpeg maintainers chose to do so. Two possible options to reduce the size of your application: (1) Instead of using ffprobe, just call "ffmpeg -i <filename>" without specifying an output file, then parse stderr: $ ffmpeg -i https://download.dolby.com/us/en/test-tones/dolby-atmos-trailer_amaze_1080.mp4 >/dev/null ffmpeg version 4.4.2 Copyright (c) 2000-2021 the FFmpeg developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://download.dolby.com/us/en/test-tones/dolby-atmos-trailer_amaze_1080.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.76.100 Duration: 00:01:03.55, start: 0.000000, bitrate: 4537 kb/s Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 128 kb/s (default) Metadata: handler_name : sound handler vendor_id : [0][0][0][0] Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 4404 kb/s, 24 fps, 24 tbr, 12288 tbn, 48 tbc (default) Metadata: handler_name : video handler vendor_id : [0][0][0][0] At least one output file must be specified This is admittedly messy compared to parsing structured ffprobe output, but it does contain all the information you mentioned (assuming duration in centiseconds is sufficiently precise for your application). (2) Link both ffmpeg and ffprobe dynamically, in which case they'll share all but a few hundred kilobytes of on-disk code. For example, consider ffmpeg and ffprobe as installed from package repositories on a variety of systems: ffmpeg 4.4.2 installed by MacPorts on macOS Monterey (x86_64): $ du -hA /opt/local/bin/ff{mpeg,probe} 339K /opt/local/bin/ffmpeg 260K /opt/local/bin/ffprobe $ diff -s <(otool -L /opt/local/bin/ffmpeg) <(otool -L /opt/local/bin/ffprobe) 1c1 < /opt/local/bin/ffmpeg: --- > /opt/local/bin/ffprobe: 72d71 < /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0) ffmpeg 5.1.2 installed from RPM Fusion on Fedora 37 (x86_64): $ du -h /usr/bin/ff{mpeg,probe} 284K /usr/bin/ffmpeg 176K /usr/bin/ffprobe $ diff -s <(ldd /usr/bin/ffmpeg | awk '{ print $1 }') <(ldd /usr/bin/ffprobe | awk '{ print $1 }') Files /dev/fd/63 and /dev/fd/62 are identical ffmpeg 4.3.5 installed from raspberrypi.org on Debian 11.6 (aarch64): $ du -h /usr/bin/ff{mpeg,probe} 276K /usr/bin/ffmpeg 176K /usr/bin/ffprobe $ diff -s <(ldd /usr/bin/ffmpeg | awk '{ print $1 }') <(ldd /usr/bin/ffprobe | awk '{ print $1 }') Files /dev/fd/63 and /dev/fd/62 are identical MinGW-w64 ffmpeg 4.4.3 installed by MSYS2 on Windows 10 (x86_64): $ du -h /mingw64/bin/ff{mpeg,probe}.exe 320K /mingw64/bin/ffmpeg.exe 184K /mingw64/bin/ffprobe.exe $ diff -s <(Dependencies -depth 1 -modules /mingw64/bin/ffmpeg.exe | tail -n +2) <(Dependencies -depth 1 -modules /mingw64/bin/ffprobe.exe | tail -n +2) Files /dev/fd/63 and /dev/fd/62 are identical ("Dependencies" is Dependencies.exe from https://github.com/lucasg/Dependencies https://github.com/lucasg/Dependencies)
chungy 4y ago
ffmpeg has got to be my favorite media tool ever. :) It has replaced all kinds of various programs I used to use, such as netpbm and imagemagick. It's just better.
eimrine 4y ago
Is it possible to mount video for Youtube using only ffmpeg for cutting footage, adding music/letters/filters and never using GUI?
- seba_dos1 4y ago
  Yes.
  - mnw21cam 4y ago
    Yes, that's how I do it. I have a script that takes a list of cuts, source videos/images, and effects, and runs ffmpeg to produce the final result. That's also how many GUI video editing tools work - they just use the GUI to build the cut list.
Gordonjcp 4y ago
I wonder how hard it is to add more decoders to it? It'd be great if it could cope with .m2t timecode from HDV video streams.
- jbk 4y ago
  Send files and bugreport
AlexeyParamonov 4y ago
Is it possible to add a label or a watermark on top of your video without re-encoding using ffmpeg?
- zweifuss 4y ago
  If you use a subtitle (or similar) as a watermark, it can be easily removed. So why should you watermark at all? So the answer is probably no.
  - AlexeyParamonov 4y ago
    Well, I am making a live Youtube stream with audio and wanted to add an album cover on top for the track when it is playing.
- pjc50 4y ago
  I don't believe that's possible with any tool since you're changing the actual video content?
  - AlexeyParamonov 4y ago
    I know some services do this somehow, so I was thinking maybe there is a way in ffmpeg.
    - pjc50 4y ago
      Hmm. Example of such a service? Wonder if we can reverse engineer it.
      - frzen 4y ago
        One that comes to mind is this AMCDX Video Patcher [1] made by Alex Mogurenko. I've played around with it and it's very cool. There is the frame editor to overlay a PNG or file to file where you can insert all or some of another video over your target file. I haven't used it in production but I've played around with it. The file to file tool seemingly only touches the effected blocks of video and there's no rendering. I just did a test where I was able to cover a persons face with an image. Alex has a video showing him blurring a section but I think blurring option was removed in favour of being able to overlay an image. [2] The video needs to be Prores and I'm not able to guarantee it's not touching any other part of the frame but it feels like it's just the effected section of the video frame that is getting changed. If you are trying it out make a copy of your original video before you start, it does the edits in place. [1] https://mogurenko.com/2021/01/29/amcdx-video-patcher-v0-6-7/ https://mogurenko.com/2021/01/29/amcdx-video-patcher-v0-6-7/ [2] https://youtu.be/T-EJRB-xAdU?t=44 https://youtu.be/T-EJRB-xAdU?t=44
jokoon 4y ago
I wrote a large and messy script to encode a random amount of videos of any aspect ratio so it can fit in a mosaic with the xstack filter. I think there are still slight audio sync issues, because I should reencode video files individually but I do a single pass instead to not deal with leftover files. I'm quite happy with the result. I wish I could write a script to do some basic effect on a video, like adding some moving text. The ideal would be to have an animated SVG file and make a video out of it.
- bencahill 4y ago
  You may want to check out AviSynth [1]. It has been quite a few years since I last used it, but it is able to do things like moving text with a script. It was very fun to work with, particularly being able to copy a script from a previous project and modify it for a new project. When I last used it, I used a text editor called AvsPmod [2] that had some nice integrations. [1] http://avisynth.nl/ http://avisynth.nl/ [2] https://avspmod.github.io/ https://avspmod.github.io/
bundie 4y ago
Awesome!
ozgurozalp 4y ago
FFmpeg is so helpful tool for editing video, I almost always use it in my projects.
ksec 4y ago
I guess we will have to wait another version for H.266 VVC support.
sorenjan 4y ago
I don't know if there's any work being done on this, but I wish ffmpeg had better support for modern ML based filters, like super resolution, frame interpolation, segmentation, automatic subtitles, etc. There was an ml filter made years ago as part of a Google summer of code projekt, which includes super resolution, but it's difficult to use and you need to train or find pretrained model files. ML is where video and audio filter research is happening at the moment, hopefully ffmpeg can get a good pipeline going. And please use an inference library that can run on all computers.
janvdberg 4y ago
I love the version names, all famous scientists (Von Neumann, Riemann, Lorentz etc.)! Though I am curious whether there is a specific reasoning behind a name in relation to a version (alphabetically, numerically, chronologically etc.)?
- gyan 4y ago
  No reasoning like that. There may be a vote if multiple candidates are proposed, each with multiple supporters.
nickjj 4y ago
I love ffmpeg, a single command of `ffmpeg -ss 01:15:42 -to 01:16:00 -i example.mp4 -c copy output.mp4` will let you create a video clip at a certain point in time from a larger video without decoding/encoding it. I wrote a blog post and made a demo video the other day going over using this feature at: https://nickjanetakis.com/blog/create-video-clips-with-ffmpeg-in-seconds https://nickjanetakis.com/blog/create-video-clips-with-ffmpe...
- vigneshv_psg 4y ago
  i understand that this is the fastest way to extract frames, but it is limited by key frame availability in the original file (i.e.) you can only start the output from a keyframe. in this case, it is important to be aware that the times you specify may not be extracted exactly. it will be off by a few frames based on keyframe availability. the only way to extract exact frames is to re-encode. :)
  - scottlamb 4y ago
    Fundamentally (ignoring ffmpeg incantations), I think there are four options: 1. re-encode the whole thing 2. re-encode just the first GOP 3. start a bit early or late 4. include the full first GOP but use an edit list to instruct the player to skip to the timestamp of interest
    - vigneshv_psg 4y ago
      yeah these are pretty much the options. i believe the ffmpeg command that OP suggested would do option 3.
  - nickjj 4y ago
    This is true. Do you have a use case where being off by a few frames might be a deal breaker? Personally I've created dozens of clips using this method and it always turns out ok. It gives you about ~1 second precision on where you want to make your cuts. After I create the clips I can play things back normally, complete with an ability to seek to specific points successfully.
c0nsumer 4y ago
Do any of you know if this has Dolby AC-4 support? Ticket 8349 [1] has been open for years to add this, but it's not there yet. This would be very nice so we can watch ATSC 3.0 OTA broadcasts via FFmpeg-based things like Plex. (Currently if one uses something like a SiliconDust HDHomeRun, viewing an ATSC 3.0 stream requires using their app/player, which uses a SiliconDust cloud service to do the decoding. It'd be really nice to have a not-network-dependent way to view/hear OTA broadcasts.) [1] https://trac.ffmpeg.org/ticket/8349 https://trac.ffmpeg.org/ticket/8349
- jdofaz 4y ago
  I was hoping for that too, but it doesn't look like it
  - c0nsumer 4y ago
    After some more digging, I'm realizing the same. Fingers crossed soon.
sylware 4y ago
I had an API breakage with audio one year ago. Is the "new" seeking API up and running? (because I have segfaults while seeking in mp4 video files with the current one), and I am ready to make the changes in my media player. Until they keep away from c++ or the ISO planned obsolescence of the C language, keep the SDK minimal, I guess I tolerate their excessive heavy use of nasm macro preprocessor. If the video is mostly about versioning, well since I use a weekly ffmpeg git...
fenesiistvan 4y ago
Is VP9 streaming already supported?

594 points by gyan 4y ago

gyan 4y ago

Changelog: * Radiance HDR image support * ddagrab (Desktop Duplication) video capture filter * ffmpeg -shortest_buf_duration option * ffmpeg now requires threading to be built * ffmpeg now runs every muxer in a separate thread * Add new mode to cropdetect filter to detect crop-area based on motion vectors and edges * VAAPI decoding and encoding for 10/12bit 422, 10/12bit 444 HEVC and VP9 * WBMP (Wireless Application Protocol Bitmap) image format * a3dscope filter * bonk decoder and demuxer * Micronas SC-4 audio decoder * LAF demuxer * APAC decoder and demuxer * Media 100i decoders * DTS to PTS reorder bsf * ViewQuest VQC decoder * backgroundkey filter * nvenc AV1 encoding support * MediaCodec decoder via NDKMediaCodec * MediaCodec encoder * oneVPL support for QSV * QSV AV1 encoder * QSV decoding and encoding for 10/12bit 422, 10/12bit 444 HEVC and VP9 * showcwt multimedia filter * corr video filter * adrc audio filter * afdelaysrc audio filter * WADY DPCM decoder and demuxer * CBD2 DPCM decoder * ssim360 video filter * ffmpeg CLI new options: -stats_enc_pre[_fmt], -stats_enc_post[_fmt], * -stats_mux_pre[_fmt] * hstack_vaapi, vstack_vaapi and xstack_vaapi filters * XMD ADPCM decoder and demuxer * media100 to mjpegb bsf * ffmpeg CLI new option: -fix_sub_duration_heartbeat * WavArc decoder and demuxer * CrystalHD decoders deprecated * SDNS demuxer * RKA decoder and demuxer * filtergraph syntax in ffmpeg CLI now supports passing file contents as option values, by prefixing option name with '/' * hstack_qsv, vstack_qsv and xstack_qsv filters

exhilaration 4y ago
This mostly looks like they're adding support for new codecs, there's only 4 new options in there, which seems rather paltry for a dot zero version release. Should we consider ffmpeg feature complete?
- gyan 4y ago
  Dot zero releases basically signify major bumps of the core libs, and are not really synced with user-facing feature changes.
- cm2187 4y ago
  I am still using 4.2. Does it make any difference if all I care about is h264 and hevc?
  - LtdJorge 4y ago
    More modern x264/x265 I guess
  - bsenftner 4y ago
    At a prior job we pinned to 4.2.3 because after that version an internal change massively impacted high frame rates. With a 1080p mp4 file on disk, and that disc being an ssd, version 4.2.3 delivers 700-800 frames per second, while all the versions after that 4.2.3 release - no other app or library changes - that same file may reach 300 fps. I spent some time trying to find the cause of that 50% cut in efficiency, but never found enough time. Curious to try this version.
    - esperent 4y ago
      > I spent some time trying to find the cause of that 50% cut in efficiency Presumably they have an issue tracker. Did you report this?
      - bsenftner 4y ago
        When I looked, it had been reported already.
        cstejerean 4y ago
        do you have a link?
    - mixmastamyk 4y ago
      You are referring to (re)encoding a file?
      - bsenftner 4y ago
FranOntanaya 4y ago
Nostalgia moment with the CrystalHD decoder deprecation. That came along the Intel Atom netbooks, one of the few ways we had to bump a bit their austere performance.

QuarterRoy 4y ago

I love software that just does what it’s supposed to do and is so good at it that it’s an underpinning for other projects. FFmpeg, curl, nmap, et., al.

Aperocky 4y ago
It is such a swiss knife kind of tool, there's just almost nothing it can't do when it comes to video, audio and image modifications. I wonder how many big companies uses it "in secret" that we don't know off, or distributes it with mentions buried deep underneath usage agreement.
- shmerl 4y ago
  Google must be using it for Youtube. Wikimedia is using it as well.
  - jzombie 4y ago
    I believe it is embedded in Chromium as well.
    - shmerl 4y ago
      Firefox bundles it as well I think (at least Linux version).
      - padenot 4y ago
        Firefox vendors a subset of ffmpeg, to decode royalty free codecs such as vp8/9, mp3, flac, on all OSes. We're using our own decoders/demuxers for other royalty free codecs for historical reasons. It can also use the system's ffmpeg to decode patented formats such as h264 or AAC, on desktop Linux, if the copy of ffmpeg present on the host includes non-free codecs (same on other OSes, with their respective system libraries).
  - komali2 4y ago
    They do, they say so here: https://support.google.com/youtube/answer/1722171#zippy=%2Ccontainer-mp%2Caudio-codec-aac-lc%2Cvideo-codec-h%2Cframe-rate%2Cbitrate%2Cvideo-resolution-and-aspect-ratio%2Ccolor-space https://support.google.com/youtube/answer/1722171#zippy=%2Cc... > YouTube does not recommend the RGB color matrix on uploads. In this case, YouTube initially sets the color matrix to unspecified before the standardization. It will then infer the color matrix using the color primaries during standardization. Note that sRGB TRC will convert to BT.709 TRC. YouTube re-tags the color primaries/matrix/TRC to BT.709 when it is not supported by FFmpeg colorspace conversion filter. Since FFmpeg is gpl2.1 I thought they had to make it easier to know they're using this, like under a "licenses" section, but I don't see anything under studio.youtube.com indicating this.

asicsp 4y ago

Changelog: https://github.com/FFmpeg/FFmpeg/blob/master/Changelog https://github.com/FFmpeg/FFmpeg/blob/master/Changelog

shmerl 4y ago

What's the story with VAAPI AV1 encoding? Does it exist anywhere or not yet?

mayli 4y ago
- VAAPI decoding and encoding for 10/12bit 422, 10/12bit 444 HEVC and VP9
- shmerl 4y ago
  That's been there for a while. AV1 decoding also exists. But I'm curious what's going on with AV1 encoding.
stonogo 4y ago
It exists only as a patch from Intel (https://github.com/intel/cartwheel-ffmpeg/blob/master/patches/0071-lavc-vaapi-support-av1-encode.patch https://github.com/intel/cartwheel-ffmpeg/blob/master/patche...) but I haven't seen any efforts to upstream it yet.
- shmerl 4y ago
  Thanks! At least changes look recent so there must be some plan to upstream it.

lxe 4y ago

Fun fact of the day: ffmpeg is written by Fabrice Bellard, who among other impressive things, wrote the JS PC emulator capable of running windows 2000 in a browser over 10 years ago that got me fascinated with emulators in general. https://bellard.org/jslinux/tech.html https://bellard.org/jslinux/tech.html

galleywest200 4y ago
He also wrote QEMU, which is probably far more impressive.
- digianarchist 4y ago
  https://smartbear.com/blog/fabrice-bellard-portrait-of-a-super-productive-pro/ https://smartbear.com/blog/fabrice-bellard-portrait-of-a-sup...
  - MuffinFlavored 4y ago
    This is missing QuickJS :) https://bellard.org/ https://bellard.org/
- mobilio 4y ago
  Also LZEXE back in time...
- xwolfi 4y ago
  Sorry, windows 98 running in JS is way way WAY more impressive. It's just insane.
- mhh__ 4y ago
  The most impressive thing is the LTE modem
Jasper_ 4y ago
Fabrice has not been involved in ffmpeg for well over 15 years now.

arjvik 4y ago

I absolutely love ffmpeg, but for the life of me I cannot understand how its pipeline system works. Each time I need to use it, I attempt to construct the command myself, but end up giving up and consulting StackOverflow. Amazingly, someone has usually done the exact thing I need to do and posted their command line to StackOverflow, so I'm never out of luck! How do I actually start understanding how ffmpeg works? I want to be an ffmpeg power user.

reidjs 4y ago
I’ve never had to directly use ffmpeg. What do you do that makes it so useful to you? Something in video production?
- pmontra 4y ago
  Not parent but in my case yes, video production: ffmpeg to extract audio from captured video, process audio with Audacity, ffmpeg to cut video and merge it with the audio from Audacity.
  - smcnally 4y ago
    Also to quickly convert video formats and containers, scale and crop renders, alter speed, make gifs
mmsimanga 4y ago
This. And imagemagick.
- Macha 4y ago
  Though I have used ffmpeg on images, too.
Aperocky 4y ago
The ordering of the arguments changes behavior, I don't think that's a good thing or desired outcome, at most it's a bug_feature. It tend to happen when big project rolls out batteries itself, in this case command line parsing.

scottlamb 4y ago

From the changelog [1]: > - DTS to PTS reorder bsf Interesting, I wonder what this is / why you'd want it. In particular, when you have the DTS but not the PTS. The recent gstreamer 1.22 release [2] had what I read as the opposite—calculate a plausible DTS from the order and PTS. They did a nice job of explaining why it's useful. AFAICT, this approach is the only viable way to get B frames to work properly from a received RTP stream. > H.264/H.265 timestamp correction elements ... Muxers are often picky and need proper PTS/DTS timestamps set on the input buffers, but that can be a problem if the encoded input media stream comes from a source that doesn't provide proper signalling of DTS, such as is often the case for RTP, RTSP and WebRTC streams or Matroska container files. Theoretically parsers should be able to fix this up, but it would probably require fairly invasive changes in the parsers, so two new elements h264timestamper and h265timestamper bridge the gap in the meantime and can reconstruct missing PTS/DTS. Looks like the ffmpeg thing is dts2pts_bsf.c. [3] I haven't really read the implementation, but I was hoping the comment at the top would illuminate things, but "Derive PTS by reordering DTS from supported streams" isn't enough for me. [1] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/release/6.0:/Changelog https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/rel... [2] https://gstreamer.freedesktop.org/releases/1.22/ https://gstreamer.freedesktop.org/releases/1.22/ [3] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/release/6.0:/libavcodec/dts2pts_bsf.c https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/rel...

jbk 4y ago
This bsf was done to fix the following ticket https://trac.ffmpeg.org/ticket/502 https://trac.ffmpeg.org/ticket/502
- scottlamb 4y ago
  Thanks. Looks like the input in question was an .avi file. That's a container format I don't know anything about (and don't particularly care to). I suspect then it's not relevant for anything more modern.
  - brigade 4y ago

FrostKiwi 4y ago

> nvenc AV1 encoding support > QSV AV1 encoder That is captial 'A' A W E S O M E. FFmpeg is always on the bleeding edge, love it.

jbk 4y ago

If you want to know a bit more about what is important in this major release, you can watch my presentation about FFmpeg 6.0 at Fosdem: https://fosdem.org/2023/schedule/event/om_vlc/ https://fosdem.org/2023/schedule/event/om_vlc/ (Video + Slides) It’s a bit less dry than the changelog, notably for the evolutions of the APIs. What’s also important is the changes about the release schedule that we’ve been pushing with the community. Major version every year at the beginning of the year, with ABI and API break, minor releases during the year and an LTS every other year…

AndyKelley 4y ago
Thank you for doing this talk. I agree with you, news about ffmpeg progress should be more widespread. And by the way, I love your hat!
vhiremath4 4y ago
Gracias!
mikae1 4y ago
At 2m15s you even mention HN. :)
- qwerty456127 4y ago
  This is very nice. I've been surprised how many people don't know HN, it can definitely use some promotion.
  - _0ffh 4y ago
    Please, no! We'd need way more dangs than are available.
    - adrian_mrd 4y ago
      From the Oxford Dictionary of Modern Slang, paperback edition from 2010 on page 65 "Dang: Expressing annoyance or antipathy, or as an intensive" ;)

ww520 4y ago

Isn’t there a language for configuring ffmpeg to make it do different things?

mixmastamyk 4y ago
The ffmpeg-python wrapper module is sorta like that.

ggm 4y ago

ffmpeg is my go-to re-encoder. I didn't know ffprobe existed until I read some "howto" and its also increadibly useful as a way to get fundamental video and audio stream data out as e.g. csv or json. So I can run an ffprobe script to get x,y info out, decide if the video needs re-encoding, pass to an ffmpeg call which does fast or veryfast settings to reset the x/y scale (for instance) It's also unquestionably 'self documenting' because all of the sheharazade 1001 options are listed in --help. The problem is knowing which one will make the horse speak.

yboris 4y ago
I've been so frustrated that FFprobe functionality is not part of FFmpeg. My app extracts screenshots from videos to create a beautiful gallery of videos. But even though I include FFmpeg already, I need a 50mb FFprobe executable to be bundled with my app just so that I can determine the width, height, duration, and fps of a video file! What is it that FFprobe does that FFmpeg couldn't do with a few extra pieces of exposed API? https://videohubapp.com/ https://videohubapp.com/ - https://github.com/whyboris/Video-Hub-App https://github.com/whyboris/Video-Hub-App https://github.com/whyboris/Video-Hub-App/blob/772b25bbd4b41be6b081e6fcb51e5babae9ae2e2/node/main-support.ts#L458 https://github.com/whyboris/Video-Hub-App/blob/772b25bbd4b41...
- jasomill 4y ago
  ffmpeg and ffprobe are built against the exact same set of libraries, so they could certainly be combined into a single program if the ffmpeg maintainers chose to do so. Two possible options to reduce the size of your application: (1) Instead of using ffprobe, just call "ffmpeg -i <filename>" without specifying an output file, then parse stderr: $ ffmpeg -i https://download.dolby.com/us/en/test-tones/dolby-atmos-trailer_amaze_1080.mp4 >/dev/null ffmpeg version 4.4.2 Copyright (c) 2000-2021 the FFmpeg developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://download.dolby.com/us/en/test-tones/dolby-atmos-trailer_amaze_1080.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.76.100 Duration: 00:01:03.55, start: 0.000000, bitrate: 4537 kb/s Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 128 kb/s (default) Metadata: handler_name : sound handler vendor_id : [0][0][0][0] Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 4404 kb/s, 24 fps, 24 tbr, 12288 tbn, 48 tbc (default) Metadata: handler_name : video handler vendor_id : [0][0][0][0] At least one output file must be specified This is admittedly messy compared to parsing structured ffprobe output, but it does contain all the information you mentioned (assuming duration in centiseconds is sufficiently precise for your application). (2) Link both ffmpeg and ffprobe dynamically, in which case they'll share all but a few hundred kilobytes of on-disk code. For example, consider ffmpeg and ffprobe as installed from package repositories on a variety of systems: ffmpeg 4.4.2 installed by MacPorts on macOS Monterey (x86_64): $ du -hA /opt/local/bin/ff{mpeg,probe} 339K /opt/local/bin/ffmpeg 260K /opt/local/bin/ffprobe $ diff -s <(otool -L /opt/local/bin/ffmpeg) <(otool -L /opt/local/bin/ffprobe) 1c1 < /opt/local/bin/ffmpeg: --- > /opt/local/bin/ffprobe: 72d71 < /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0) ffmpeg 5.1.2 installed from RPM Fusion on Fedora 37 (x86_64): $ du -h /usr/bin/ff{mpeg,probe} 284K /usr/bin/ffmpeg 176K /usr/bin/ffprobe $ diff -s <(ldd /usr/bin/ffmpeg | awk '{ print $1 }') <(ldd /usr/bin/ffprobe | awk '{ print $1 }') Files /dev/fd/63 and /dev/fd/62 are identical ffmpeg 4.3.5 installed from raspberrypi.org on Debian 11.6 (aarch64): $ du -h /usr/bin/ff{mpeg,probe} 276K /usr/bin/ffmpeg 176K /usr/bin/ffprobe $ diff -s <(ldd /usr/bin/ffmpeg | awk '{ print $1 }') <(ldd /usr/bin/ffprobe | awk '{ print $1 }') Files /dev/fd/63 and /dev/fd/62 are identical MinGW-w64 ffmpeg 4.4.3 installed by MSYS2 on Windows 10 (x86_64): $ du -h /mingw64/bin/ff{mpeg,probe}.exe 320K /mingw64/bin/ffmpeg.exe 184K /mingw64/bin/ffprobe.exe $ diff -s <(Dependencies -depth 1 -modules /mingw64/bin/ffmpeg.exe | tail -n +2) <(Dependencies -depth 1 -modules /mingw64/bin/ffprobe.exe | tail -n +2) Files /dev/fd/63 and /dev/fd/62 are identical ("Dependencies" is Dependencies.exe from https://github.com/lucasg/Dependencies https://github.com/lucasg/Dependencies)

chungy 4y ago

ffmpeg has got to be my favorite media tool ever. :) It has replaced all kinds of various programs I used to use, such as netpbm and imagemagick. It's just better.

eimrine 4y ago

Is it possible to mount video for Youtube using only ffmpeg for cutting footage, adding music/letters/filters and never using GUI?

seba_dos1 4y ago
Yes.
- mnw21cam 4y ago
  Yes, that's how I do it. I have a script that takes a list of cuts, source videos/images, and effects, and runs ffmpeg to produce the final result. That's also how many GUI video editing tools work - they just use the GUI to build the cut list.

Gordonjcp 4y ago

I wonder how hard it is to add more decoders to it? It'd be great if it could cope with .m2t timecode from HDV video streams.

jbk 4y ago
Send files and bugreport

AlexeyParamonov 4y ago

Is it possible to add a label or a watermark on top of your video without re-encoding using ffmpeg?

zweifuss 4y ago
If you use a subtitle (or similar) as a watermark, it can be easily removed. So why should you watermark at all? So the answer is probably no.
- AlexeyParamonov 4y ago
  Well, I am making a live Youtube stream with audio and wanted to add an album cover on top for the track when it is playing.
pjc50 4y ago
I don't believe that's possible with any tool since you're changing the actual video content?
- AlexeyParamonov 4y ago
  I know some services do this somehow, so I was thinking maybe there is a way in ffmpeg.
  - pjc50 4y ago
    Hmm. Example of such a service? Wonder if we can reverse engineer it.
    - frzen 4y ago
      One that comes to mind is this AMCDX Video Patcher [1] made by Alex Mogurenko. I've played around with it and it's very cool. There is the frame editor to overlay a PNG or file to file where you can insert all or some of another video over your target file. I haven't used it in production but I've played around with it. The file to file tool seemingly only touches the effected blocks of video and there's no rendering. I just did a test where I was able to cover a persons face with an image. Alex has a video showing him blurring a section but I think blurring option was removed in favour of being able to overlay an image. [2] The video needs to be Prores and I'm not able to guarantee it's not touching any other part of the frame but it feels like it's just the effected section of the video frame that is getting changed. If you are trying it out make a copy of your original video before you start, it does the edits in place. [1] https://mogurenko.com/2021/01/29/amcdx-video-patcher-v0-6-7/ https://mogurenko.com/2021/01/29/amcdx-video-patcher-v0-6-7/ [2] https://youtu.be/T-EJRB-xAdU?t=44 https://youtu.be/T-EJRB-xAdU?t=44

jokoon 4y ago

I wrote a large and messy script to encode a random amount of videos of any aspect ratio so it can fit in a mosaic with the xstack filter. I think there are still slight audio sync issues, because I should reencode video files individually but I do a single pass instead to not deal with leftover files. I'm quite happy with the result. I wish I could write a script to do some basic effect on a video, like adding some moving text. The ideal would be to have an animated SVG file and make a video out of it.

bencahill 4y ago
You may want to check out AviSynth [1]. It has been quite a few years since I last used it, but it is able to do things like moving text with a script. It was very fun to work with, particularly being able to copy a script from a previous project and modify it for a new project. When I last used it, I used a text editor called AvsPmod [2] that had some nice integrations. [1] http://avisynth.nl/ http://avisynth.nl/ [2] https://avspmod.github.io/ https://avspmod.github.io/

bundie 4y ago

Awesome!

ozgurozalp 4y ago

FFmpeg is so helpful tool for editing video, I almost always use it in my projects.

ksec 4y ago

I guess we will have to wait another version for H.266 VVC support.

sorenjan 4y ago

I don't know if there's any work being done on this, but I wish ffmpeg had better support for modern ML based filters, like super resolution, frame interpolation, segmentation, automatic subtitles, etc. There was an ml filter made years ago as part of a Google summer of code projekt, which includes super resolution, but it's difficult to use and you need to train or find pretrained model files. ML is where video and audio filter research is happening at the moment, hopefully ffmpeg can get a good pipeline going. And please use an inference library that can run on all computers.

janvdberg 4y ago

I love the version names, all famous scientists (Von Neumann, Riemann, Lorentz etc.)! Though I am curious whether there is a specific reasoning behind a name in relation to a version (alphabetically, numerically, chronologically etc.)?

gyan 4y ago
No reasoning like that. There may be a vote if multiple candidates are proposed, each with multiple supporters.

nickjj 4y ago

I love ffmpeg, a single command of `ffmpeg -ss 01:15:42 -to 01:16:00 -i example.mp4 -c copy output.mp4` will let you create a video clip at a certain point in time from a larger video without decoding/encoding it. I wrote a blog post and made a demo video the other day going over using this feature at: https://nickjanetakis.com/blog/create-video-clips-with-ffmpeg-in-seconds https://nickjanetakis.com/blog/create-video-clips-with-ffmpe...

vigneshv_psg 4y ago
i understand that this is the fastest way to extract frames, but it is limited by key frame availability in the original file (i.e.) you can only start the output from a keyframe. in this case, it is important to be aware that the times you specify may not be extracted exactly. it will be off by a few frames based on keyframe availability. the only way to extract exact frames is to re-encode. :)
- scottlamb 4y ago
  Fundamentally (ignoring ffmpeg incantations), I think there are four options: 1. re-encode the whole thing 2. re-encode just the first GOP 3. start a bit early or late 4. include the full first GOP but use an edit list to instruct the player to skip to the timestamp of interest
  - vigneshv_psg 4y ago
    yeah these are pretty much the options. i believe the ffmpeg command that OP suggested would do option 3.
- nickjj 4y ago
  This is true. Do you have a use case where being off by a few frames might be a deal breaker? Personally I've created dozens of clips using this method and it always turns out ok. It gives you about ~1 second precision on where you want to make your cuts. After I create the clips I can play things back normally, complete with an ability to seek to specific points successfully.

c0nsumer 4y ago

Do any of you know if this has Dolby AC-4 support? Ticket 8349 [1] has been open for years to add this, but it's not there yet. This would be very nice so we can watch ATSC 3.0 OTA broadcasts via FFmpeg-based things like Plex. (Currently if one uses something like a SiliconDust HDHomeRun, viewing an ATSC 3.0 stream requires using their app/player, which uses a SiliconDust cloud service to do the decoding. It'd be really nice to have a not-network-dependent way to view/hear OTA broadcasts.) [1] https://trac.ffmpeg.org/ticket/8349 https://trac.ffmpeg.org/ticket/8349

jdofaz 4y ago
I was hoping for that too, but it doesn't look like it
- c0nsumer 4y ago
  After some more digging, I'm realizing the same. Fingers crossed soon.

sylware 4y ago

I had an API breakage with audio one year ago. Is the "new" seeking API up and running? (because I have segfaults while seeking in mp4 video files with the current one), and I am ready to make the changes in my media player. Until they keep away from c++ or the ISO planned obsolescence of the C language, keep the SDK minimal, I guess I tolerate their excessive heavy use of nasm macro preprocessor. If the video is mostly about versioning, well since I use a weekly ffmpeg git...

fenesiistvan 4y ago

Is VP9 streaming already supported?