8 ms·
FFmpeg devs boast of another 100x leap thanks to handwritten assembly code
- ivanjermakov 1y agoRelated: ffmpeg's guide to writing assembly: https://news.ycombinator.com/item?id=43140614 https://news.ycombinator.com/item?id=43140614
- 486sx33 1y ago[dead]
- shmerl 1y agoStill waiting for Pipewire + xdg desktop portal screen / window capture support in ffmpeg CLI. It's been dragging feet forever with it.
- askvictor 1y ago[flagged]
- pizlonator 1y agoThe hardest part of optimizations like this is verifying that they are correct. We don’t have a reliable general purpose was of verifying if any code transformation is correct. LLMs definitely can’t do this (they will lie and say that something is correct even if it isn’t).
- viraptor 1y agoBut we do! For llvm there's https://github.com/AliveToolkit/alive2 https://github.com/AliveToolkit/alive2 There are papers like https://people.cs.rutgers.edu/~sn349/papers/cgo19-casmverify-preprint.pdf https://people.cs.rutgers.edu/~sn349/papers/cgo19-casmverify... There's https://github.com/google/souper https://github.com/google/souper There's https://cr.yp.to/papers/symexemu-20250505.pdf https://cr.yp.to/papers/symexemu-20250505.pdf And probably other things I'm not aware of. If you're limiting the scope to a few blocks at a time, symbolic execution will do fine.
- pizlonator 1y ago> limiting the scope to a few blocks Yeah so like that doesn’t scale. The interesting optimizations involve reasoning across thousands of blocks And my point is there is no reliable general purpose solution here. „Only works for a few blocks at a time” is not reliable. It’s not general purpose
- viraptor 1y agoNo optimisation currently done in production compilers (that I know of) work on more than a few blocks. I haven't seen anyone claim they're not general purpose or don't scale yet, or are not reliable. Meanwhile the patch we're discussing is 2 functions, 3 blocks each. So in this context we're not talking about anything crazy.
- pizlonator 1y agoExamples of optimization passes in llvm that reason across arbitrary numbers of blocks: - SROA - GVN - regalloc And those are just the heaviest hitters. In fact most optimization passes are capable of performing changes that impact many blocks at once.
- viraptor 1y agoEh, kinda? I mean regalloc is not an optimisation pass. And SROA/GVM go through whole functions, but they don't really optimise across blocks. They do local changes only with information from previous blocks available. I meant that I don't know of anything that makes modifications across multiple blocks as one complex change. Even licm pokes one thing at a time from one block to a different one. There's no complex "let's replace this kind of cross-block pattern with a different equivalent code that changes multiple blocks at the same time".
- pizlonator 1y agoSROA absolutely does a coordinated modification across an arbitrary number of blocks whenever it kills an alloca. GVN has cases where it’ll also make a coordinated change across multiple blocks And those are just two passes. I’m not mentioning the many other passes that have this property just for the sake of brevity.
- LtWorf 1y ago> I wonder how many optimisations like this could be created by LLMs Zero. There's no huge corpus of stackoverflow questions on highly specific assembly optimisations so…
- astrange 1y agoYou can run an agent in a loop, but for something this small you can already use a SAT solver or superoptimizer if you want to get out of the business of thinking about things yourself. I've never seen anyone actually do it, mostly because modeling the problem is more work than just doing it.
- ksclk 1y ago> you can already use a SAT solver Could you elaborate please? How would you approach this problem, using a SAT solver? All I know is that a SAT solver tells you whether a certain formula of ANDs and ORs is true. I don't know how it could be useful in this case.
- dragontamer 1y agoPretty much all instructions at the assembly level are sequences of AND/OR/XOR operations. SAT solvers can prove that some (shorter) sequences are equivalent to other (longer) sequences. But it takes a brute force search. IIRC, these super optimizing SAT solvers can see patterns and pick 'Multiply' instructions as part of their search. So it's more than traditional SAT. But it's still... At the end of the day.... A SAT equivalence problem.
- peaboff 1y agoWhat you're saying is true. Yes you can grind away at generating sequences of instructions, SAT solve equivalence and benchmark, but of course, you would sooner see all black holes in the observable universe evaporate before you find an instruction sequence that is both correct AND 100x faster.
- 1y ago
- deleted 1y ago[deleted]
- viraptor 1y agohttps://arxiv.org/html/2505.11480v1 https://arxiv.org/html/2505.11480v1 we're getting there. This is for general purpose code, which is going to be easier than heavy SIMD where you have to watch out for very specific timing, pipelines and architecture details. But it's a step in that direction.
- smj-edison 1y agoI'd be worried about compile times, lol. Final binaries are quite often tens to hundreds of megabytes, pretty sure an LLM processes tokens much slower than a compiler completes passes. EDIT: another thought: non-deterministic compilation would also be an issue unless you were tracking the input seed, and it would still cause spooky action at a distance unless you had some sort of recursive seed. Compilers are supposed to be reliable and deterministic, though to be fair advanced optimizations can be surprising because of the various heuristics.
- viraptor 1y agoThere's no reason to run the optimisation discovery at compile time. Anything that changes the structure can be run to change the source ahead of time. Anything that doesn't can be generalised into a typical optimisation step in the existing compiler pipeline. Same applies to Souper for example - you really don't want everyone to run it.
- smj-edison 1y agoI'm not quite understanding your comment, are you saying that ANNs are only useful for tuning compiler heuristics?
- viraptor 1y agoThat too. But mainly for transforming the source ahead of time to be more optional. If there's some low level, local optimisation, that can be found once and implemented as a stable, repeatable idea in the compiler code instead.
- Arubis 1y agoThis intrinsically feels like the opposite of a good use case for an LLM for code gen. This isn’t boilerplate code by any means, nor would established common patterns be helpful. A lot of what ffmpeg devs are doing at the assembly level is downright novel.
- hashishen 1y agoIt doesn't matter. This is inherently better because the dev knows exactly what is being done. Llms could cripple entire systems with assembly access
- gronglo 1y agoYou could run it in a loop, asking it to improve the code each time. I know what the ffmpeg devs have done is impressive, but I would be curious to know if something like Claude 4 Opus could make any improvements.
- minimaxir 1y agoThat can work with inefficient languages like Python, but not raw Assembly.
- saagarjha 1y agoSure they can, it’s just that verifying the outputs is difficult.
- eukara 1y agoI think if it was easy for them to improve critical projects like ffmpeg, we'd have seen some patches that mattered already. The only activity I've seen is LLMs being used to farm sec-ops bounties which get rejected because of poor quality. https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-for-intelligence/ https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-f...
- hansvm 1y agoIf we're considering current-gen LLMs, approximately zero. They're bad at this sort of thing even with a human in the loop.
- Aardwolf 1y agoThe article somtimes says 100x, other times it says 100% speed boost. E.g. it says "boosts the app’s ‘rangedetect8_avx512’ performance by 100.73%." but the screenshot shows 100.73x. 100x would be a 9900% speed boost, while a 100% speed boost would mean it's 2x as fast. Which one is it?
- pizlonator 1y agoThe ffmpeg folks are claiming 100x not 100%. Article probably has a typo
- k_roy 1y agoThat would be quite the percentage difference with 100x
- MadnessASAP 1y ago100x to the single function 100% (2x) to the whole filter
- torginus 1y agoI'd guess the function operates of 8 bit values judging from the name. If the previous implementation was scalar, a double-pumped AVX512 implementation can process 128 elements at a time, making the 100x speedup plausible.
- ErrorNoBrain 1y ago[flagged]
- ethan_smith 1y agoIt's definitely 100x (or 100.73x) as shown in the screenshot, which represents a 9973% speedup - the article text incorrectly uses percentage notation in some places.
- andersmurphy 1y agoThe article probably contains traces of blue.
- pavlov 1y agoOnly for x86 / x86-64 architectures (AVX2 and AVX512). It’s a bit ironic that for over a decade everybody was on x86 so SIMD optimizations could have a very wide reach in theory, but the extension architectures were pretty terrible (or you couldn’t count on the newer ones being available). And now that you finally can use the new and better x86 SIMD, you can’t depend on x86 ubiquity anymore.
- Aurornis 1y agoAVX512 is a set of extensions. You can’t even count on an AVX512 CPU implementing all of the AVX512 instructions you want to use, unless you stick to the foundation instructions. Modern encoders also have better scaling across threads, though not infinite. I was in an embedded project a few years ago where we spent a lot of time trying to get the SoC’s video encoder working reliably until someone ran ffmpeg and we realized we could just use several of the CPU cores for a better result anyway
- AaronAPU 1y agoWhen I spent a decade doing SIMD optimizations for HEVC (among other things), it was sort of a joke to compare the assembly versions to plain c. Because you’d get some ridiculous multipliers like 100x. It is pretty misleading, what it really means is it was extremely inefficient to begin with. The devil is in the details, microbenchmarks are typically calling the same function a million times in a loop and everything gets cached reducing the overhead to sheer cpu cycles. But that’s not how it’s actually used in the wild. It might be called once in a sea of many many other things. You can at least go out of your way to create a massive test region of memory to prevent the cache from being so hot, but I doubt they do that.
- torginus 1y agoSorry for the derail, but it sounds like you have a ton of experience with SIMD. Have you used ISPC, and what are your thoughts on it? I feel it's a bit ridiculous that in this day and age you have to write SIMD code by hand, as regular compilers suck at auto-vectorizing, especially as this has never been the case with GPU kernels.
- almostgotcaught 1y ago> Have you used ISPC No professional kernel writer uses Auto-vectorization. > I feel it's a bit ridiculous that in this day and age you have to write SIMD code by hand You feel it's ridiculous because you've been sold a myth/lie (abstraction). In reality the details have always mattered.
- CyberDildonics 1y agoISPC is a lot different from C++ compiler auto vectorization and it works extremely well. Have you tried it or not? If so where does it actually fall down? It warns you when doing slow stuff like gathers and scatters.
- camel-cdr 1y agoIs it a lot different from autovec with #pragma omp simd? I played around with ISPC a bit and it didn't seem much different.
- jauntywundrkind 1y agoKind of reminds me of Sound Open Firmware (SOF), which can compile with e8ther unoptimized gcc, or using the proprietary Cadence XCC compiler that can can use the Xtensa HiFi SIMD intrinsics. https://thesofproject.github.io/latest/introduction/index.html#toolchain-faq https://thesofproject.github.io/latest/introduction/index.ht...
- tombert 1y agoActually a bit surprised to hear that assembly is faster than optimized C. I figured that compilers are so good nowadays that any gains from hand-written assembly would be infinitesimal. Clearly I'm wrong on this; I should probably properly learn assembly at some point...
- mhh__ 1y agoCompilers are extremely good considering the amount of crap they have to churn through but they have zero information (by default) about how the program is going to be used so it's not hard to beat them.
- haiku2077 1y agoIf anyone is curious to learn more, look up "profile-guided optimization" which observes the running program and feeds that information back into the compiler
- deleted 1y ago[deleted]
- mananaysiempre 1y agoLooking at the linked patches, you’ll note that the baseline (ff_detect_range_c) [1] is bog-standard scalar C code while the speedup is achieved in the AVX-512 version (ff_detect_rangeb_avx512) [2] of the same computation. FFmpeg devs prefer to write straight assembly using a library of vector-width-agnostic macros they maintain, but at a glance the equivalent code looks to be straightforwardly expressible in C with Intel intrinsics if that’s more your jam. (Granted, that’s essentially assembly except with a register allocator, so the practical difference is limited.) The vectorization is most of the speedup, not the assembly. To a first approximation, modern compilers can’t vectorize loops beyond the most trivial (say a dot product), and even that you’ll have to ask for (e.g. gcc -O3, which in other cases is often slower than -O2). So for mathy code like this they can easily be a couple dozen times behind in performance compared to wide vectors (AVX/AVX2 or AVX-512), especially when individual elements are small (like the 8-bit ones here). Very tight scalar code, on modern superscalar CPUs... You can outcode a compiler by a meaningful margin, sometimes (my current example is a 40% speedup). But you have to be extremely careful (think dependency chains and execution port loads), and the opportunity does not come often (why are you writing scalar code anyway?..). [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346725.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346725.h... [2] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346726.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346726.h...
- cpncrunch 1y agoArticle is unclear what will actually be affected. It mentions "rangedetect8_avx512" and calls it an obscure function. So, what situations is it actually used for, and what is the real-time improvement in performance for the entire conversion process?
- brigade 1y agoIt's not conversion. Rather, this filter is used for video where you don't know whether the pixels are video or full range, or whether the alpha is premultiplied, and determining that information. Usually so you can tag it correctly in metadata. And the function in question is specifically for the color range part.
- cpncrunch 1y agoIt's still unclear from your explanation how it's actually used in practice. I run thousands of ffmpeg conversions every day, so it would be useful to know how/if this is likely to help me. Are you saying that it's run once during a conversion as part of the process? Or that it's a specific flag that you give, it then runs this function, and returns output on the console? (Either of those would be a one-time affair, so would likely result in close to zero speed improvement in the real world).
- brigade 1y agoThis is a new filter that hasn’t even been committed yet, it only runs if explicitly specified, and would only ever be specified by someone that already knows that they don’t know the characteristics of their video. So you wouldn’t ever run this.
- cpncrunch 1y agoThank you, exactly what I was looking for.
- nwallin 1y agoBack in ye olden tymes, video was an analog signal. It was wibbly wobbly waves. You could look at them with an oscilloscope. One of the things that it used to do to make stuff work was to encode control stuff in band. This is sorta like putting editor notes in a text file. There might be something like <someone should look up whether or not this is actually how editor's notes work>. In particular, it used blacks which were blacker than black to signal when it was time to go to the next line, or to go to the next frame. In later times, we developed DVDs. DVDs were digital. But they still had to encode data that would ultimately be sent across an analog cable to an analog television that would display the analog signal. So DVDs used colors darker than 16 (out of 255) to denote blacker than black. This digital signal would be decoded to an analog signal and were sent directly onto the wire. So while DVDs are ostensibly 8 bit per channel color, it's more like 7.9 bits per channel. This is also true for BluRay and HDMI. In more recent times, we've decided we want that extra 0.1 bits back. Some codecs will encode video that uses the full range of 0-255 as in-band signal. The problem is that ... sometimes people do a really bad job of telling the codec whether the signal range is 0-255 or 16-255. And it really does make a different. Sometimes you'll be watching a show or movie or whatever and the dark parts will be all fucked up. There are several reasons this can happen, one of which is because the black level is wrong. It looks like this function determines scans frames for whether all the pixels are in the 16-255 or 0-255 range. If a codec can be sure that the pixel values are 16-255, it can saves some bits will encoding. But I could be wrong. I do video stuff at my day job, and much to my own personal shame, I do not handle black levels correctly.
- deleted 1y ago[deleted]
- jabjoe 1y agoInteresting. The one time I ended up writing assembly was because of SIMD. I was speaking about it recently and had forgotten it was because SIMD instructions, so nice to be reminded. However, that one time, I also ended up finding the right syntactic sugar, to get the compiler to do it right. If I remember right it was all down to aliasing. I had to convince the compiler the data wasn't going to be accessed any other place. It wasn't working that out itself, so it didn't know it could use the SIMD instruction I was. Once I found this and the right non-standard extra key words, the compiler would do it right. So I ended removing the assembly I had written.
- bravesoul2 1y ago100x * * not 100x
- blueflow 1y agoHeadline: "FFmpeg devs boast of another 100x leap thanks to handwritten assembly code" Text: "... this boost is only seen in an obscure filter", "... up to ... %" [expletives omitted]
- Ygg2 1y agoTo be fair to the dev he said he sped up one function 100x. That doesn't imply he speed up the program. Ironically speeding up parts of code may decrease overall performance due to resource contention. As the saying goes. There are lies, damned lies, statistics, and then benchmarking.