7 ms·
I'm writing some tools to capture video streams, doing demuxing/muxing, decoding, etc. All of the libraries that do these things are written in C. Using anothe
by dmm 11y ago
I'm writing some tools to capture video streams, doing demuxing/muxing, decoding, etc. All of the libraries that do these things are written in C.
Using another language, aside from c++, would involve using some kind of ffi mapping, made by me or someone else. So now I not only have to understand the semantics of c and the libraries I'm using I have to understand the semantics of the ffi mapping.
Using the ffmpeg libraries I've had to dig into the source to figure out what it's doing[0]. If I'm using python(or whatever) now I have to do that and figure out what the mapping is doing. And I'm still not safe from segfaults and memory leaks, they're just harder to find and understand.
Soft realtime is trivial in C. If I'm capturing a video stream at 10fps I have about 100ms to do everything to handle a frame. This is very easy to do in c. If I'm using a garbage collected language I have 100ms-X, where X is the longest garbage collection pause.
I tried doing a similar project in java and found it very difficult. Using any library, for example to output a jpeg, created tons of garbage, resulting in long gc pauses. I tried a bunch of different -XXasdf options that helped but to really solving the problem would involve reimplementing a bunch of stuff to reuse buffers and avoid garbage. I freely admit this may be because of my ignorance of how to do it correctly but again it's trivial to avoid these pauses in c.
So I ask sincerely what should I do? I'll gladly throw out the years of painfully acquired c knowledge if you have a better solution for me today.
[0] I wonder why avformat_find_stream_info is detecting the from framerate? Good luck :)
http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source.html#l03123 http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source...
- lmm 11y agoIt's not a great situation yeah - I'd heard there was some effort on Ocaml multimedia libraries but I don't know what's happened to that. Mozilla appear to have got some of their video stack implemented in rust but nowhere near the range of what ffmpeg supports. Hopefully in a few years we'll get there, but in the meantime there's no way around running some C code. Even so, I would (and have) still use an FFI and put as much as possible in a safer language. You will get segfaults and memory leaks because you're still calling C code, but you won't be adding more, and IME that's well worth the overhead of an FFI. (Also ffmpeg has had quite a bit of attention from people using valgrind, afl-fuzz etc, so its defect rate is lower than your C code would likely have, unless you're also using those tools). Alternatively, depending on your use case you might be able to separate the decoding/encoding stages from the processing, in which case you can write the code without needing much in the way of libraries, and pipe to/from ffmpeg. Write a quick prototype and see if you can get the performance you need within GC or a small amount of -XXasdf (you might be surprised, particularly with the state of Haskell and OCaml today. I've done literally this from Java, JNI-ing the ffmpeg libraries to do realtime streaming, and had it work fine, and that was five years ago). In the rare case something like that doesn't cut it Rust is what I'd use (there are other options but I know less about them).