12 ms·

FFmpeg School of Assembly Language

• netr0ute 2y ago
  The only thing I don't like about this is the focus on x86 assembly, which is a sinking ship because RISC-V is coming to eat its lunch, FAST.
  • KeplerBoy 2y ago
    I could understand if you wrote arm, because that's an architecture with actual marketshare. arguably more marketshare than x86-64 at this point, but you had to choose risc-v for the lols.
  • high_na_euv 2y ago
    HackerNews does not reflect real world well
    • ksec 2y ago
      The unwritten rule of HN: You do not criticise The Rusted Holy Grail and the Riscy Silver Bullet.
  • do_not_redeem 2y ago
    How would you define "fast"?
    • hagbard_c 2y ago
      In relative terms, compared with similarly priced and powered devices on the market. RISC-V does lag behind the others - ARM, x86/64 - here, at least for now.
    • snvzz 2y ago
      Not eating. Only drinking water or zero calory drinks such as black coffee. Only while fasting can a person think clearly. When thinking clearly, RISC-V is inevitably chosen as the ISA. Fasting will also eventually make you hungry. Thus "RISC-V is coming to eat its lunch, FAST."
  • wolf550e 2y ago
    Where are the high performance RISC-V implementations? Those that compete with AMD Zen-5 and Apple M4? Or at least AWS Graviton 4?
    • zozbot234 2y ago
      The Tenstorrent folks are working on that.
  • 201984 2y ago
    Wake me up when a RISC-V processor is on par with an N50.
  • deleted 2y ago
    [deleted]
  • astrange 2y ago
    Doesn't RISC-V use vector stream processing instead of SIMD? That's a poor fit for ffmpeg.
    • astrange 2y ago
      I should say, I think it would be. I haven't actually tried it and know ARM has added it too, so it'd be interesting to see for sure.
• xuhu 2y ago
  "Assembly language of FFmpeg" leads me to think of -filter_complex. It's not for human consumption even once you know many of its gotchas (-ss and keyframes, PTS, labeling and using chain outputs, fading, mixing resolutions etc). But then again no-one is adjusting timestamps manually in batch scripts, so a high-level script on top of filter_complex doesn't have much purpose.
  • pdyc 2y ago
    what do you mean by no purpose? you can adjust them programmatically in batch scripts.
  • chgs 2y ago
    I use filter-complex all the time, often in batch scripts
• slicktux 2y ago
  Kudos for the K&R reference! That was the book I bought to learn C and programming in general. I had initially tried C++ as my first language but I found it too abstract to learn because I kept asking what was going on underneath the hood.
• agumonkey 2y ago
  I remember kempf saying most of recent development on codecs is in raw asm. Only logical that they can write some tutorials :)
• henning 2y ago
  This is what Hacker News should be about. Awesome. Thank you.
• lukaslalinsky 2y ago
  This is perfect. I used to know the x86 assembly at the time of 386, but for the more advanced processors, it was too complex. I'd definitely like to learn more about SIMD on recent CPUs, so this seems like a great resource.
• philip22 2y ago
  [dead]
• imglorp 2y ago
  Asm is 10x faster than C? That was definitely true at some point but is it still true today? Have compilers really stagnated so badly they can't come close to hand coded asm?
  • jki275 2y ago
    Probably some very niche things. I know I can't write ASM that's 10x better than C, but I wouldn't assume no one can.
    • 1propionyl 2y ago
      It depends on what you're trying to do. I would in general only expect such substantial speedups when considering writing computation kernels (for audio, video, etc). Compilers today are liable in most circumstances to know many more tricks than you do. Especially if you make use of hints (e.g. "this memory is almost always accessed sequentially", "this branch is almost never taken", etc) to guide it.
      • jki275 2y ago
        Oh I definitely agree that in the vast majority of cases the compiler will probably win. But I suspect there are cases where the super experts exist who can do things better.
      • astrange 2y ago
        Mm, those hints don't matter on modern CPUs. There's no good way for the compiler to pass it down to them either. There are some things like prefetch instructions, but unless you know the exact machine you're targeting, you won't know when to use them.
    • CyberDildonics 2y ago
      It isn't very hard to write C that is 10x better than C, because most programs have too many memory allocations and terrible memory access patterns. Once you sort that out you are already more than 10x ahead, then you can turn on the juice with SIMD, parallelization and possibly optimize for memory bandwidth as well.
• sylware 2y ago
  A gigantic mistake was done in much of ffmpeg assembly: They are abusing nasm macro-preprocessor up to obscene levels...
  • ryanianian 2y ago
    Why is it "abusing," and what would you suggest as an alternative?
    • sylware 2y ago
      Have a look an their code, it is obvious. Often you have to figure out what actually the macros does, and I remember it was not that straight forward. And the macro language is specific to nasm. What to do: unroll the macros and/or use a little abstraction using a simple common macro preprocessor, aka not tied to the assembler. And I am just doing exactly that: my x86_64 assembly code does assemble with fasm/nasm/gas with a little abstraction using a C preprocessor.
      • pengaru 2y ago
        there is nothing wrong with depending on nasm
        • sylware 2y ago
          Yes, it is since you can with a little C preprocessor abstraction assemble with fasm/gas/nasm.
      • PhilipRoman 2y ago
        To be fair, nasm allows you to detach the preprocessor from the assembler (-E). But I agree with you in general.
• eachro 2y ago
  This looks great! Is there going to be exercises or a project based component as well?
• belter 2y ago
  Uhmmm...Lots of praise but these are just three small lessons covering basics. Exercises not uploaded yet. Looks like a work in progress or in the beginning?
• thayne 2y ago
  It doesn't mention the downsides of using assembly. The biggest of which is that your code is architecture specific, so for example you have to write different code for x86 and arm, and possibly even different code for x86_64. Unfortunately, for SIMD, there isn't really a great way to write portable code for it, at least in C. Rust is working on stabilizing a portable simd API, and zig has simd support, but I suspect ffmpeg would still complain they aren't quite as fast as they would like. One thing that confuses me is the opposition to inline asm. It seems like inline asm would be more efficient than having to make a function call to an asm function.
  • arkj 2y ago
    If you look at this from a top-down perspective, you’ll see downsides, but from a bottom-up view, those same differences can be an advantage. Different architectures have different capabilities, and writing assembly means you’re optimizing for performance rather than prioritizing code portability or maintenance.
  • hereonout2 2y ago
    Possibly they could have added that warning, but at the same time this is a guide from the ffmpeg project, presumably for ffmpeg developers. They lay it out quite clearly I think, but things like libavcodec are probably one of the few types of project where the benefits of assembly outweigh the lack of portability. I'm not sure rust or zig's support for SIMD would be the project's first complaint either. Likely more concerned with porting a 25 year old codebase to a new language first.
  • wffurr 2y ago
    What about Highway? https://github.com/google/highway https://github.com/google/highway I suppose that's C++ not C though.
    • kccqzy 2y ago
      I've enjoyed using Highway, but it does in fact use plenty of C++ features that would make it unappealing to C projects. And if you make even just one mistake, it's easy to get several screenfuls of error messages; I accept that as a C++ developer but C developers would hate it.
• neallindsay 2y ago
  Things that you would expect every software developer to know today will one day become niche, low-level knowledge.
  • jesserobbins 2y ago
    Everything is an abstraction :-)
• mkoubaa 2y ago
  I'm shocked there still isn't a hardware accelerator for video decoding.
  • Narishma 2y ago
    What do you mean? Pretty much any SoC designed for consumer applications has some form of hardware accelerated video decoding.
  • graypegg 2y ago
    There is! FFmpeg supports hardware acceleration for a lot of operations. (Though format/codec dependant on the chipset you're working with, so it's not as general as you might expect. I don't know a ton about video's guts, so I assume the variance between video codec decoding is big enough to require incompatible special silicon.) https://trac.ffmpeg.org/wiki/HWAccelIntro https://trac.ffmpeg.org/wiki/HWAccelIntro
  • ghhrjfkt4k 2y ago
    ffmpeg does more than hardware decoding. For example scaling, cropping, changing colors, effects. All this stuff can benefit from vectorized operations (on CPU or GPU).
  • qingcharles 2y ago
    Practically every computer device manufactured in the last 15 years has some sort of accelerator/specific instructions designed primarily for optimizing the decoding of video.
• Daniel_Van_Zant 2y ago
  I'm curious from anyone who has done it. Is there any "pleasure" to be had in learning or implementing assembly (like there is for LISP or RISC-V) or is it something you learn and implement because you want to do something else (like learning COBOL if you need to work with certain kinds of systems). It has always piqued my interest but I don't have a good reason in my day-to-day job to get into it. Wondering if it is worth committing some time to for the fun of it.
  • daeken 2y ago
    I have spent the last ~25 years deep in assembly because it's fun. It's occasionally useful, but there's so much pleasure in getting every last byte where it belongs, or working through binaries that no one has inspected in decades, or building an emulator that was previously impossible. It's one of the few areas where I still feel The Magic, in the way I did when I first started out.
  • ghhrjfkt4k 2y ago
    I once used it to get a 4x speedup of sqrt computations, by using SIMD. It was quite fun, and also quite self contained and manageable. The library sqrt handles all kinds of edge-cases which prevent the compiler from autovectorizing it.
  • msaltz 2y ago
    I did the first 27 chapters of this tutorial just because I was interested in learning more and it was thoroughly enjoyable: https://mariokartwii.com/armv8/ https://mariokartwii.com/armv8/ I actually quite like coding in assembly now (though I haven’t done much more than the tutorial, just made an array library that I could call from C). I think it’s so fun because at that level there’s very little magic left - you’re really saying exactly what should happen. What you see is mostly what you get. It also helped me understand linking a lot better and other things that I understood at a high level but still felt fuzzy on some details. Am now interested to check out this ffmpeg tutorial bc it’s x86 and not ARM :)
    • Daniel_Van_Zant 2y ago
• fracus 2y ago
  I'm halfway through this tutorial and I'm really enjoying it. I haven't touched assembly since back in university decades ago. I've always had an urge to optimize processes for some reason. This scratches that itch. I was also more curious about SIMD since hearing about it on Digital Foundry.
• jupp0r 2y ago
  I personally don't think there's much value in writing assembly (vs using intrinsics), but it's been really helpful to read it. I have often used Compiler Explorer (https://godbolt.org/ https://godbolt.org/) to look at the assembly generated and understand optimizations that compilers perform when optimizing for performance.
  • frontfor 2y ago
    Your commented is directly contradicted by the article. > To make multimedia processing fast. It’s very common to get a 10x or more speed improvement from writing assembly code, which is especially important when wanting to play videos in real time without stuttering.
    • TinkersW 2y ago
      They said they prefer intrinsics which the article says are only about 10% slower(citation needed), you misunderstood and made a comparison against scalar. Personally I'd say the only good reason to use assembly over intrinsics is having control over calling convention, for example the windows CC is absolute trash and wastes many SIMD registers.
    • edward28 2y ago
      And how often are you doing multimedia processing?
• ej1 2y ago
  This os a great article!
• jancsika 2y ago
  What's the cost of shuttling data in and out of SIMD land?
  • School-Cotton 2y ago
    SIMD doesn’t operate on a separate memory space or anything like that. You just load data from normal memory into the SIMD registers, just like you would have to load it into the scalar registers if you wanted to operate on it with normal instructions.
    • aidenn0 2y ago
      On some targets you need to overalign data for vectorization.
    • astrange 2y ago
      It is slow to move data from SIMD to scalar registers, or can be.
      • TinkersW 2y ago
        It depends, for SIMD float-> scalar floats it is fast as they operate on the same registers. If pulling out of lane 0 you don't even need to do anything(just a type cast). For other lanes you need a shuffle. For SIMD integer to scalar integer, it has to move into separate register, so there is some short penalty(3 cycles iir).
  • kccqzy 2y ago
    It's pretty cheap. You can easily find the latency and throughput numbers on different Intel architectures. Here's an example for movdqa: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#ig_expand=4029,0,4029&text=_mm_load_si128 https://www.intel.com/content/www/us/en/docs/intrinsics-guid... which is a basic 128-bit load. Even a 512-bit load isn't much more expensive: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm512_load_epi64&ig_expand=4006,3993 https://www.intel.com/content/www/us/en/docs/intrinsics-guid...
• foresto 2y ago
  > Note that the “q” suffix refers to the size of the pointer *(*i.e in C it represents *sizeof(*src) == 8 on 64-bit systems, and x86asm is smart enough to use 32-bit on 32-bit systems) but the underlying load is 128-bit. I find that sentence confusing. I assume that i.e is supposed to be i.e., but What is *(* supposed to mean? Shouldn't that be just an open parenthesis? In what context would *sizeof(*src) be considered valid? As far as I know, sizeof never yields a pointer. I get the impression that someone sprinkled random asterisks in that sentence, or maybe tried to mix asterisks-denoting-italics with C syntax.
  • kevingadd 2y ago
    Yes, this looks like something went wrong with the markdown itself or the conversion of the source material to markdown.
  • sweeter 2y ago
    Wouldn't it return the size of the pointer? I would guess it's exclusively used to handle architecture differences
    • foresto 2y ago
      Strictly speaking, or maybe just the way I personally think of it, sizeof doesn't return anything. It's not a function, so it doesn't return at all. (At least, not at run time.) Nitpicking aside, the result of sizeof(*src) would be the size of the object at which the pointer points. The type of that result is size_t. That's what makes this code from the lesson I quoted invalid: *sizeof(*src) That first asterisk tries to dereference the result of sizeof as though it were a pointer, but it's a size_t: an unsigned integer type. Not a pointer.
      • sweeter 2y ago
        Yea but that first asterisk is incorrect
        • foresto 2y ago
• computerbuster 2y ago
  Another resource on the same topic: https://blogs.gnome.org/rbultje/2017/07/14/writing-x86-simd-using-x86inc-asm/ https://blogs.gnome.org/rbultje/2017/07/14/writing-x86-simd-... As I'm seeing in the comments here, the usefulness of handwritten SIMD ranges from "totally unclear" to "mission critical". I'm seeing a lot on the "totally unclear" side, but not as much on the "mission critical", so I'll talk a bit about that. FFmpeg is a pretty clear use case because of how often it is used, but I think it is easier to quantify the impact of handwriting SIMD with something like dav1d, the universal production AV1 video decoder. dav1d is used pretty much everywhere, from major browsers to the Android operating system (superseding libgav1). A massive element of dav1d's success is its incredible speed, which is largely due to how much of the codebase is handwritten SIMD. While I think it is a good thing that languages like Zig have built-in SIMD support, there are some use cases where it becomes necessary to do things by hand because even a potential performance delta is important to investigate. There are lines of code in dav1d that will be run trillions of times in a single day, and they need to be as fast as possible. The difference between handwritten & compiler-generated SIMD can be up to 50% in some cases, so it is important. I happen to be somewhat involved in similar use cases, where things I write will run a lot of times. To make sure these skills stay alive, resources like the FFmpeg school of assembly language are pretty important, in my opinion.
  • zbobet2012 2y ago
    So on point. We do _a lot_ of hand written SIMD on the other side (encoders) as well for similar reasons. In addition on the encoder side it's often necessary to "structure" the problem so you can perform things like early elimination of loops, and especially loads. Compilers simply can not generate autovectorized code that does those kinds of things.
  • dundarious 2y ago
    What does Zig offer in the way of builtin SIMD support, beyond overloads for trivial arithmetic operations? 90% of the utility of SIMD is outside of those types of simple operations. I like Zig, but my understanding is you have to reach for CPU specific builtins for the vast majority of cases, just like in C/C++. GCC and Clang support the vector_size attribute and overloaded arithmetic operators on those "vectorized" types, and a LOT more besides -- in fact, that's how intrinsics like _mm256_mul_ps are implemented: `#define _mm256_mul_ps(a,b) (__m256)((v8sf)(a) * (v8sf)(b))`. The utility of all of that is much, much greater than what's available in Zig.
• krick 2y ago
  Huh, I didn't even know ffmpeg still actively employs assembly in its source code.
• wruza 2y ago
  I don’t care about the split, just wanted to say that this guide is so good. I wish I had this back when I was interested in low-low-level.
• toisanji 2y ago
  Just wondering, would it make sense to use LLMs to translate higher level languages to assembly or to directly write in assembly?
  • mikestew 2y ago
    Are you asking if an LLM can produce better assembly than an optimizing compiler?
  • saagarjha 2y ago
    Generally, no. It is hard to do this translation in a way that is correct, much less performant.
• kierank 2y ago
  I am the author of these lessons. Ask me anything.
  • christiangenco 2y ago
    Hacker News is such a cool website. Hi thank you for writing this!
  • cnt-dracula 2y ago
    Hi, thanks for your work! I have a question, as someone who can just about read assembly but still do not intuitively understand how to write or decompose ideas to utilise assembly, do you have any suggestions to learn / improve this? As in, at what point would someone realise this thing can be sped up by using assembly? If one found a function that would be really performant in assembly how do you go about writing it? Would you take the output from a compiler that's been converted to assembly or would you start from scratch? Does it even matter?
    • qingcharles 2y ago
      You're looking for the tiniest blocks of code that are run an exceptional number of times. For instance, I used to work on graphics renderers. You'd find the bit that was called the most (writing lines of pixels to the screen) and try to jiggle the order of the instructions to decrease the number of cycles used to move X bits from system RAM to graphics RAM. When I was doing it, branching (usually checking an exit condition on a loop) was the biggest performance killer. The CPU couldn't queue up instructions past the check because it didn't know whether it was going to go true or false until it got there.
      • booi 2y ago
        Don’t modern or even just not ancient cpus use branch prediction to work past a check knowing that the vast majority of the time the check yields the same result?
        • 2y ago
• Charon77 2y ago
  This is very approachable and beginner friendly. Kudos to authors.
• beebaween 2y ago
  I'm kind of stunned we haven't gotten something better / more rust based than ffmpeg? Especially curious given the advent of apple metal etc. Does anyone have recommendations?
  • adamnemecek 2y ago
    Why? It's a Herculean effort. It took it was 28 years between the creation of C and FFMPEG, so if there still is not a replacement by 2038, then your complaint is justified.
  • mvdtnz 2y ago
    "Rust based" is not a feature. End users DO NOT CARE. What's your value prop?
    • filleduchaos 2y ago
      Gstreamer is increasingly developed in Rust, and is a far saner, better documented and more flexible framework for developers than libav/ffmpeg. The pipeline/plugin based architecture is pretty neat even as an end user, I find it a lot more discoverable.
      • lukaslalinsky 2y ago
        Gstreamer is a high level API that uses FFmpeg, not a FFmpeg replacement.
        • filleduchaos 2y ago
          This is just blatantly wrong, as anyone that has actually looked at the project's documentation and code can tell. Because GStreamer is designed to be modular and plugin-based, all sorts of plugins exist for it (including libav plugins) but it is itself a media framework that is quite capable of replacing ffmpeg. Neither is "higher level" than the other.
• imchaz 2y ago
  I'll be honest, I didn't read through much. Ffmpeg gives me severe ptsd. My first task out of college was to write a procedurally generated video using ffmpeg, conform to dash, and get it under 150kb/s while being readable. Docs were unusable. Dash was only a few months old. And stackoverflow was devoid of help. I kid you not, the only way to get any insight was some sketchy IRC channel. (2016 btw, well past IRCs prime)
  • thegrim33 2y ago
    Not trying to be too negative but the memories your comment brought up in me, I need to rant about ffmpeg for a minute. ffmpeg is the worst documented major library I've ever used in my life. I integrated with it to render videos inside my 3D engine and boy do I shiver at any thought of having to work with it again. The "documentation" is a collect of 15-20 year old source samples. The vast majority of them either won't compile anymore because the API has changed, or they use 2, 3, or 4 times deprecated functions that shouldn't be used anymore. The source examples also have almost no comments explaining anything. They have super dense, super complicated code, with no comments, but then there will be a line like "setRenderSpeed(3)" or whatever and it'll have a comment: "Sets render speed to 3", the absolute most useless comment ever. The source examples are also all written in 30 year old as C-Style of C code as you can get, incredibly horribly dense, with almost no organization, have to jump up and down all over the file to find the global variables being accessed, it's just gross and barely comprehensible. They put a lot of effort into producing doxygen documentation for everything, but the doxygen documentation is nearly useless, it just lists the API with effectively zero documentation or explanation on the functions or parameters. There's so little explanation of how to do anything. On the website they have sections for each library, and for most libraries you get 2-3 sentences of explanation on what the library is for, and that's it. That's the extent the entire library is documented. They just drop an undocumented massive C API split across a dozen or so libraries on you and wish you luck. The API has also gotten absolutely wrecked over the last 20 years or however long it's been around as it has evolved. Sometimes they straight up delete functions to deprecate them, sometimes they create a new version of a function as fuction2 and then as function3, and keep all of them around, sometimes they replace a function with a completely differently named function and keep them both around, and there's absolutely nothing written anywhere about what the "right" way to do anything is, what functions you should actually be using. So many times I went down rabbit holes reading some obscure 15 year old mailing list post trying to find anyone that had successfully done something I was trying to do. And again, the obscure message board posts and documentation that does exist is almost all deprecated at this point and shouldn't be used. Then there's the custom build system, so if you need to build it custom to support or not support different features, you can't use any modern build system, it's all custom scripts that do weird things like hardcoded dumping build output into your home directory. Makes it difficult to integrate with a modern build system. It has so much momentum, and so many users, but man, there has to be a massive opening for someone to replace ffmpeg with a modern programming language and a modern build system, built with GPU acceleration of stuff in mind from the beginning and not tacked on top 20 years later, and not using 30 year old c-style code, and an actually documented project.
• fulafel 2y ago
  SIMD was introduced in the 80s but become ubiquitous when Intel got in on it in the 90s. It's interesting that (for x86), PLT is still stuck at hand-writing assembly 40 years later.
• buserror 2y ago
  I used to do quite a bit of SIMD version of critical functions, but now I rarely do -- one thing to try is isolate that code, and run it in the Most Excellent Compiler Explorer [0]. And stare at the generated code! More often than not, the auto-vectorisation now generates pretty excellent SIMD version of your function, and all you have to do is 'hint' the compiler -- for example explicitly list alignment, provide your own vector source/destination type -- you can do a lot by 'styling' your C code while thinking about what the compiler might be able to do with it -- for example, use extra intermediary variables, really break down all the operations you want etc. Worst case if REALLY the compiler isn't clever enough, this give you a good base to adapt the generated assembly to tweak, without having to actually write the boilerplate bits. In most case, the resulting C function will be vectorized as good, or better than the hand coded one I'd do -- and in many other cases, it's "close enough" not to matter that much. The other good news is that that code will probably vectorize fine for WASM and NEON etc without having to have explicit versions. [0] https://godbolt.org/ https://godbolt.org/
  • anonymoushn 2y ago
    I have no idea how to get the compiler to generate wider-than-16 pshufb in the general case, for example, and for the 16-wide case, writing the actual definition of pshufb prevents you from getting pshufb while writing a version with UB gets you pshufb.
  • kimixa 2y ago
    We did something slightly similar - for the very few isolated things it makes sense (e.g. image up/download and conversions in the gpu driver that weren't supported/large enough to be worth firing off a gpu job to complete), they were initially written in C and used the compiler annotations to specify things like the alignment or allowed pointer aliasing in order to make it generate the code wanted. GCC and Clang both support some vector extensions, that allow somewhat portable implementations of things like scatter-gather, or shuffling things around or masking elements in a single register that's hard to specify clearly enough so that it's both readable for humans and will always generate the expected code between compiler versions in "plain" C. But due to needing to support other compilers and platforms we actually ended up importing the generated asm from those source files in the actual build.