7 ms·
A Journey Through Color Space with FFmpeg
- brucethemoose2 4y agoYou should check out VapourSynth, which makes colorspace conversion tasks like this very organized and Pythonic: https://www.vapoursynth.com/doc/functions/video/resize.html https://www.vapoursynth.com/doc/functions/video/resize.html But it also sometimes suffers from trying to parse the original colorspace, whether you use use ffmpeg reader or not.
- dylan604 4y ago"What is color space 1? And what are primaries?" Be thankful you have the ability to set the color space and primaries within the command. Back in the days of dinosaurs (mid-2000s to 2010ish) and when HD was just an infant sucking its thumb and before Apple updated the MOV container to hold these values, it was even more annoying to get colors correct. The big issue then was gamma settings. Windows used one value and Macs used another, and there were always issues getting them to look the same on either platform. There was also the bad ol' days of when iTunes would reject your MOVs for not having these primaries and color space values in your files created by Final Cut Pro. It turns out, that within the silos of Apple, they do no communicate back and forth with each other. So the iTunes team read the white paper on the MOV file format and wrote all of their automation expecting these values to be present. Unfortunately, the team writing FCP had not updated their exporter to use these new features of the MOV. So iTunes rejected the files from not being written by an official Apple product!?!? something about left hand ands right hands can be written here. To the point, we had to have a 3rd party app developed to take the files from FCP and update them to be compliant with iTunes. I still have scars from the shrapnel from those "early" days. If only articles like this were made at the beginning of the features rather than 15+ years after the fact and can be written after doing a google search. So hopefully future me types can find this rather than the hell of needing the feature and having to get the feature developed, and then just try it and hope for the best.
- aikinai 4y agoThis is a shot in the dark, but you seem very knowledgeable about color, video formats, and the Apple ecosystem so I thought you might have some insight into this issue I'm facing recently. I'm trying to process HLG3 video files from my Sony camera and have spent far too many hours trying to figure out how different settings affect QuickTime, ffmpeg, and Davinci Resolve, among others. The biggest thing I can't figure out right now is why Davinci Resolve renders the HLG video brighter than QuickTime. If I enable 10-bit precision and Mac display color profiles for the Davinci viewer, it does seem to tap into the HDR capabilities on my MacBook and allow for full dynamic range that looks similar to QuickTime's render. However, everything is brighter, especially highlights. Do you know of any good references to better understand how HDR works on a modern Mac?
- dylan604 4y agoI got out of this particular job type just before HDR became widespread as a consumer product. Chasing color accuracy is one of the top 3 things I hate about computing only proceeded by choosing a font, and printing. I spent time in a color correction facility where we had to calibrate Sony studio reference CRTs, LCDs (or other flat panel tech), Christie projectors, and desktop monitors. I absolutely hate Hate HATE it. The external monitors being fed by a Decklink card would have to be calibrated using a dedicated instrument for each type of screen. The desktop screens would have to be calibrated by another device. At the end of the day, the client was always advised to base color decisions on the calibrated display monitors and NOT the desktop monitors. Luckily, all of this was handled by the video engineering department, and I only know as much about it from what I asked. Thankfully, it was never my responsibility! So, I guess I'm saying good luck with that! ;-) Trying to dial in all of the different display devices to match is a journey into madness. There was a time when commercial content was only ever critically evaluated on broadcast quality monitors. As web content became larger, people were less concerned about the broadcast safe and pushed values to no longer be broadcast safe so that they looked good on iPhones. All of that to say, you're not alone in your pain. Also, are you working a PC or Mac? If you're on a PC, I have zero experience with this on that platform. Fun fact (using air quotes), there was a bug in QTPlayer so that the first file you opened would display with an incorrect gamma value. When doing critical color decisions with that player, we would open some random file first, then open up the rest of the files we were interested in. Don't know if that bug is still around or not. If you're facing brightness differences between apps, my first assumption would be gamma differences between apps.
- ggm 4y agoI generally hate analogies, and I welcome being told this analogy is wrong: you know that thing where ads on radio "sound louder" but when you hassle the station they swear blind the total db on the signal as "volume" remains constant? Thats what "colourspace" is to me, in sound: the colours add up to "white" but how you got there alters what it looks like. That, plus people respond differently (rods and cones) to different specific intensities coming in. Total energy? same-same. Human response? Not same.
- pavlov 4y agoA color space also defines what “white” means, not just the interpretation of the component values. For example, sRGB defines a D65 white point. This is the absolute color of the light that the monitor should emit for white pixels. It’s usually defined using the XYZ color space (or derived x/y chromaticity coordinates). XYZ is the scientific baseline for color, but it’s not widely used for content creation. Other color spaces are designed to make color more intuitive and easier to encode. Talking about color in the XYZ space would be sort of like talking about music using absolute frequencies. It’s often more practical and flexible to say “note A” rather than “440 Hz.”
- Sniffnoy 4y ago> I like to use the mnemonics S=short, M=medium, and L=large wavelengths to remember all of this. That's not a mnemonic, that's what they actually stand for...
- sylware 4y agoI use my own-coded ffmpeg based video player and since video color spaces are a gigantic mess, I don't use the vulkan hardware for that, but convert via ffmpeg the video color space directly to my vulkan framebuffer color space. Still, I would use the vulkan 2D scaler with filtering.
- PaulHoule 4y agoI think it’s funny because I have gotten into red-cyan anaglyph stereograms both for screen and print and found that the more “advanced” hardware and techniques I use the worse results I get. https://en.wikipedia.org/wiki/Anaglyph_3D https://en.wikipedia.org/wiki/Anaglyph_3D I have a high color gamut Dell monitor and what it does it takes (0, 255, 0) in sRGB space and turn it into something like (16, 186, 15) because the primaries are a little more saturated than the sRGB primaries. You wouldn’t want sRGB pure green turned into “hit by a green laser pointer green” would you? The trouble is red got mixed in with my green so now there is a “ghost” image in the wrong eye. If I view an image in a tk canvas on Windows, tk does no color correction so I don’t see the ghosts, but if I use a modern application like Photoshop or a web browser it does color correction and I do see them. I know I could get better results if I made an image in a high color gamut space but haven’t yet operationalized this because I’m more interested in print than screen at the moment. I was thinking pretty seriously of making a WebGL viewer that does the anaglyph processing in the browser but that’s a non-starter if I can only output in the sRGB color space. As for print I have the same problems, I know I get worse results if I load a color profile for the specific paper I use and have Photoshop manage colors than if I set the printer to use ICM color calibration. I’m in the middle right now of setting up some controlled experiments that let me measure the ghosting caused by color “correction” which is not a “correction” at all for stereograms.
- mncharity 4y ago> making a WebGL viewer that does the anaglyph processing in the browser but that’s a non-starter if I can only output in the sRGB color space. There's apparently now WebGL support for display-p3[1] on Chrome and Safari.[2] Anaglyph shaders can be straightforward[3] - I had fun playing with shallow-3D desktop UIs. [1] https://en.wikipedia.org/wiki/DCI-P3 https://en.wikipedia.org/wiki/DCI-P3 [2] https://caniuse.com/?search=drawingBufferColorSpace https://caniuse.com/?search=drawingBufferColorSpace https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/drawingBufferColorSpace https://developer.mozilla.org/en-US/docs/Web/API/WebGLRender... https://bugzilla.mozilla.org/show_bug.cgi?id=1771373 https://bugzilla.mozilla.org/show_bug.cgi?id=1771373 [3] https://www.shadertoy.com/results?query=anaglyph https://www.shadertoy.com/results?query=anaglyph https://threejs.org/examples/webgl_effects_anaglyph.html https://threejs.org/examples/webgl_effects_anaglyph.html
- crazygringo 4y agoThis is a great article, not only in explaining a lot of aspects of color spaces, but also highlighting just how maddening it is to use tools that deal with color spaces, because every tool has all sorts of silent defaults and assumptions that are always different/incompatible with another tool. It's maddening. And it's only getting worse now with HDR (high dynamic range) content, as opposed to regular SDR (standard dynamic range). Fun fact: on a Mac at least, if you have the same content (hit TV show or movie) in SDR (usually 8-bit 1080p) and HDR (usually 10-bit 4K), there is no video player that will play the HDR version in a way that matches the SDR version. They range from unwatchable (VLC plays HDR content impossibly dark) to watchable-but-still-darker (Infuse) to weird (IINA is normal brightness but the colors have weird tint/saturation). How HDR is defined in theory sounds reasonable but is disastrous in practice. While SDR just maps brightness to whatever your display is (from 0 to 255), HDR maps actual real-life brightness from pitch black to the brightest sun (from 0 to 1024). So the idea is that if you shoot outdoors on the brightest day, a hypothetical screen someday should be able to display it equally bright. Now this is disastrous in practice from a creative standpoint because if you're watching a movie, cutting from a dark indoor nighttime bedroom scene to a sunny outdoor party scene is going to hurt your eyes. It's too jarring. Movies and TV generally don't want a real-life range of brightness. Not to mention, of course, that we don't have display technologies that can achieve it anyways, or even anything close to it. So what are TV shows and movies doing? As far as I can tell, they're essentially filming and color-correcting in SDR as they've always done, and then using some totally arbitrary formula to convert that to HDR for HDR mastering. Bright sunlight scenes are never encoded as actual sunlight brightness (thank god), the relative difference between a nighttime bedroom scene and a bright outdoor scene in HDR is the same as it is in SDR (again, thank god). The only HDR "special features" seem to be that things like VFX lightsabers, explosions, sparks, and so forth get extra-boosted brightness in HDR. Now back to the video players on Macs: because there is no official recommendation of how to translate HDR to real screens that have traditionally only shown SDR content (e.g. a Macbook display), they all do it differently. Which sucks. Not only are they all playing HDR content badly, there isn't even a right way to do it. We're lost in the desert. What's the solution? Somebody needs to contact Hollywood to figure out what the de-facto translation is that they're using when they release SDR and HDR versions of the same content, and then just run the formula backwards so playing my TV show that's HDR-mastered looks identical to the SDR source they also put out. (At least for regular scenes, both nighttime bedroom and outdoor party. The VFX things like lightsabers and explosions might get washed out to bright white but that's far preferable to the other 99% of the content being too dark.)