9 ms·

Adventures with ffmpeg and color ranges

177 points by doppp 7y ago

brownbat 7y ago
Interesting find. I have encoded video for over a decade, and I don't think a two month period has gone by where I haven't learned something that convinces me to completely change how or what flags I'm using. This is way beyond that but still.
- PorterDuff 7y ago
  At the risk of telling you something you know, there's a lot more things to trip on with proper color space conversion than just going from scaled to full range color in 601. It's something of rat's nest to cover all the cases.
  - dylan604 7y ago
    After ProRes became a thing, there was a lot of confusion where the metadata saved within the MOV started to include the HD vs SD color primaries. Some programs were able to correctly read this data. Most did not. Most programs did not include the data including Final Cut. The iTunes team started rejecting files exported from FCP because they claimed they were not made correctly and not a valid file. The iTunes team were just reading the spec and built their automation based on that, but not what the real world output from FCP was doing.
VectorLock 7y ago
Its weird to think how John Carmack went from .plan files to Facebook Posts. I think it might be safe to say he had the most widely read .plan file in existence. Wish he would go back to it.
- jakear 7y ago
  I suspect it’s not his decision to make.
  - pvg 7y ago
    You think someone is telling Carmack where he can and can't post? Why one earth would they do that? And why on earth would he put up with it?
    - setpatchaddress 7y ago
      It doesn't have to be coercive. Some of us would like to know what the people who hire us are up to even if we don't personally have any interest in it.
    - jakear 7y ago
      More that the company’s culture says you should post things using the company’s platform, and to not do so might be making a statement he doesn’t want to make. Otherwise, why would he move away from .plan?
      - tqi 7y ago
        Could it be that a lot of people don't know what a .plan file is, or how to read content from one?
        roghummal 7y ago
        If John Carmack started updating a .plan, people would figure out how to check a .plan.
      - pvg 7y ago
        You said 'it wasn't his decision to make' not 'perhaps he wants to use his employer's platform'. If you don't want to imply it's coercive, you should phrase it in a way that doesn't imply it's coercive.
        jakear 7y ago
        I believe it’s a “perhaps he feels he should be using his employer’s platform”. A murky coercion middle ground.
- kwijibob 7y ago
  Yes I wondered this also. It is naturally to use his employers platform. However I wonder if in 10-15 years this post will be lost in Facebook changes, but the .plan files from the 1990s will still be available somewhere.
  - enneff 7y ago
    Just as the .plan files were mirrored by some enthusiasts, so too will his Facebook posts.
    - whatshisface 7y ago
      Are they mirrored already? I don't want to visit facebook.
      - hoseja 7y ago
        In this very comment section. https://news.ycombinator.com/item?id=20036710 https://news.ycombinator.com/item?id=20036710
- deleted 7y ago
  [deleted]
- ekianjo 7y ago
  Maybe he was asked to publish all his communication on Facebook as part of his career move. Along with a large compensation. Unless he really believes Facebook is the right place to post this kind of things nowadays, who knows?
ajxs 7y ago
Funnily enough, I actually ended up giving myself a crash course in this very topic when encoding frag movies for Quake 3/Quake Live. It was/is common practice to use 'brightskins' on your enemy models in competitive play to make them easier to see, typically being a colour like '#00FF00'. I noticed that when uploading to online media hosting sites like Youtube these bright colours would become noticeably washed out and degraded. My investigation of why this happened led me all the way down this rabbit hole. This was the better half of ten years ago now so I have no idea if it still occurs, but I ended up just rendering the videos using a better brightskin colour that appeared less degraded after conversion... Going through the cycle of 'rerender/reupload/check' to find a replacement colour again and again to experiment sure was a major pain!
zaroth 7y ago
> This is usually visible as ugly banding or blocking in dark scenes. I was just complaining about this a couple weeks ago with some friends after the “Dark” episode of GoT. I was saying the codecs don’t seem to have caught up or they are just being fed at too low a bit rate to really excel at dark scenes. It’s hard to imagine this is just a scaling issue with the color ranges, and if so, how could it possibly not have been fixed?! I have a high end OLED screen in a very dark room, and I was very disappointed by the picture quality. Enough to wish I hadn’t watched it through HBO Go and that I had a better source. I would love to see some before/after scenes showing the effect of the patch.
- brigade 7y ago
  It's not; full-range YCbCr doesn't exist anywhere in the "pro" or broadcast pipelines that would have been used for GoT. I think he meant to say that the limited dynamic range of 8-bit video manifests most obviously in dark scenes, since that's where standard gamma curves are the most imprecise relative to our vision (which was fine back when TVs couldn't reproduce that range anyway.) The 14% from full-range doesn't help that much; really you need the 4x finer precision you get from 10bit.
- brokenmachine 7y ago
  >I have a high end OLED screen in a very dark room, and I was very disappointed by the picture quality. Unfortunately for you, high end OLED in a very dark room is literally the most likely conditions under which you'd be annoyed by these problems... :-) HDR is the way to go.
  - zaroth 7y ago
    It’s Dolby Vision from the ATV to the set, it’s the source encoding I assume that just couldn’t live up to material. Perhaps bitrate compromises have to be made when you’re streaming to so many people at once, but it’s not usually so glaring.
    - brokenmachine 7y ago
airstrike 7y ago
http://archive.is/K5Fpv http://archive.is/K5Fpv
- modriano 7y ago
  Thank you. I really didn't want to go on Facebook.
- nvartolomei 7y ago
  TIL: EE, a mobile provider, blocks archive.is as unsafe content for people under 18. https://twitter.com/nvartolomei/status/1133644305812381696 https://twitter.com/nvartolomei/status/1133644305812381696
  - cr3ative 7y ago
    That sounds correct to me, since it could contain NSFW content, and by default, mobile providers in England have content controls switched on.
- __HYde 7y ago
  http://archive.fo/K5Fpv http://archive.fo/K5Fpv > .IS domain will likely stop working soon[0] [0] https://blog.archive.today/ https://blog.archive.today/
forgotmypw5 7y ago
John Carmack 7 hrs · Adventures with ffmpeg and color ranges. A video legacy issue that can cause a lot of problems is the issue of “limited” versus “full” component range. For arcane reasons related to TV broadcast limitations, many video formats restrict the YUV color components to be in the 16..235 or 16..240 range instead of the full 0..255 range. Losing 14% of the already-barely-enough 8 bit dynamic range is bad enough, but it also often results in the black range starting at a quite visible grey value because most players don’t rescale the range. This is usually visible as ugly banding or blocking in dark scenes. For ffmpeg, the trick to avoid this is to use ‘-pix_fmt yuvj420p’, which says to use the j-for-jpeg full range in a 420 YUV subsampled p-for-planar format. If you are starting with either RGB images, a 10/12 bit format, or a yuvj420p format video as input, then with the libx264 codec, you would get a full range output. Note that any video processing tools used along the way could also limit the range, and once it is gone, there is no getting it back, so you must be very careful and check your entire pipeline! When you us this format, ffmpeg complains about ‘deprecated pixel format used, make sure you did set range correctly’, but you should ignore this warning. Ffmpeg would like the world to move to specifying the range independently from the YUV channel subsampling and layout: Setting ‘-color_range 0 -pix_fmt yuv420p’ makes the output format yuv420p Setting ‘-color_range 1 -pix_fmt yuv420p’ makes the output format yuv420p(tv) Setting ‘-color_range 2 -pix_fmt yuv420p’ makes the output format yuv420p(pc) Unfortunately, this isn’t yet uniformly handled throughout all the internal format tests. The libx265 integration in ffmpeg didn’t support the deprecated yuvj420p pixel format, only the basic yuv420p one, and no matter what I did, my test videos were always coming out limited range. It didn’t matter if you add a ’-color_range 2’, or ‘-x265-params range=full’. Those will change the settings in the VUI (Video Usability Information) section of the output, but the values are still compressed to the limited range. Regardless of the input data, any 8 bit h265 video coming out of ffmpeg was limited range! I walked through the libx265 code looking for range compression, but it turned out that all the damage was being done by ffmpeg before it got to x265. Ffmpeg will automatically convert formats when the output differs from the input, and since x265 only supported yuv420p, any full range input will be processed. Adding ‘-v 48’ to ffmpeg will dump more information, including this auto_scaler invocation, which is what is killing the full color range: [auto_scaler_0 @ 000001f55ecd1a40] w:2048 h:2048 fmt:bgr24 sar:0/1 -> w:2048 h:2048 fmt:yuv420p sar:0/1 flags:0x4 I was about to start hacking the code to at least do what I wanted for my use case, but I tried an appeal to Twitter: https://twitter.com/ID_AA_Carmack/status/1131715388067274753 https://twitter.com/ID_AA_Carmack/status/1131715388067274753 My suspicions were confirmed, but Derek Buitenhuis went ahead and submitted an official patch to get yuvj420p accepted by libx265, and windows builds are already available at https://ffmpeg.zeranoe.com/builds/ https://ffmpeg.zeranoe.com/builds/. I suspect there was probably some way of working around this involving explicit format conversion filters with -src_range and -dst_range overrides, but this is now working as you would expect it: ffmpeg -i source.mp4 -c:v libx265 -pix_fmt yuvj420p dest.mp4 Bravo to ffmpeg!
carmacks_fb 7y ago
The best part about these blog posts is that people discuss the hosting service more than the content of the article at this point. No one is nearly as interested in Carmack's thoughts on codecs as they are about the domain name of the server he's posting to. And yet, we'd all read his stuff no matter where he puts it. But maybe he reaches more outsiders this way? I doubt it.
dearrifling 7y ago
Fun fact: sRGB also has a non-black black point that is (fortunately) virtually ignored by everyone.
- devit 7y ago
  Source? As far as I can tell in XYZ black is (0,0,0) and the only sRGB value that maps to that is (0,0,0)
  - klodolph 7y ago
    This is incorrect. If sRGB (1,1,1) is XYZ (1,1,1), then sRGB (0,0,0) is XYZ (0.0025,0.0025,0.0025). Or if you are using absolute XYZ values, sRGB (0,0,0) is XYZ (0.1901,0.2,0.2178).
- klodolph 7y ago
  The sRGB black point just means that (0,0,0) is assumed to have a luminance of 0.2 cd/m^2, because your screen reflects ambient light and other factors. Ignoring it will give you nonsensical values when you try to do conversions into other color spaces or when you try to do color comparisons.
slavik81 7y ago
The ffmpeg documentation for YouTube videos[1] suggests using `-pix_fmt yuv420p` with H.264 and there are warnings against using other pixel formats (such as yuv444p) for compatibility reasons. Should the documentation instead be suggesting `-pix_fmt yuvj420p`? What are the risks associated with choosing that output format? [1]: https://trac.ffmpeg.org/wiki/Encode/YouTube https://trac.ffmpeg.org/wiki/Encode/YouTube
- _Gyan_ 7y ago
  No, limited range is the most common and expected range for YUV videos. Many players don't look or respect the container metadata such as nclx in MP4 or the bitstream VUI flags and assume limited range.
  - bscphil 7y ago
    This is correct. In fact, the following statement from the post is nonsense: > For arcane reasons related to TV broadcast limitations, many video formats restrict the YUV color components to be in the 16..235 or 16..240 range instead of the full 0..255 range. Losing 14% of the already-barely-enough 8 bit dynamic range is bad enough, but it also often results in the black range starting at a quite visible grey value because most players don’t rescale the range. This is usually visible as ugly banding or blocking in dark scenes. Of course players "rescale" the range. (It's not exactly rescaling, it's just that a value of 235 IS reference white in Rec. 709.) In fact, the thought if them not doing this is kind of crazy, as Rec. 709 (used in Blurays and elsewhere) as well as Rec. 601 explicitly use limited range YUV. Players that don't use limited range correctly would be out of spec and couldn't play a single Bluray disc correctly. So for the overwhelming majority of use cases, you want regular old yuv420p. The author is picking up on something that actually happens, however. There are Blurays out there that have incorrectly converted YUV values. This happens when for example something is in limited range YUV, and then that data is passed to a converter that treats the data as full range YUV, and compresses it again to fit in limited range YUV. This "doubly compressed" data is embarrassingly common (maybe 5% or so of Blurays have this problem). If you get a Bluray like this, or any video with the same issue, the blacks will be bright grays and the whites will be dull (similar to how the author describes limited range YUV). The player on the end is reading the YUV values correctly, but the data has been doubly compressed so it only undoes one round of this. The issue is well known in the pirate communities as an example. Reliable release groups always fix these broken Blurays in their encodes. This is usually described as "fixing levels" or something like that.
hiisukun 7y ago
Not sure if this kind of meta-comment is appropriate, but I find it difficult to follow a link like this because of the lack of human readable information contained in the URL. If the story was hosted on a personal blog I recognised (like Julia Evans to take a random example), then perhaps the domain would be enough for me to choose to read the article to see if the content is interesting. If the link was to a broader domain that hosts many writers, I might use the parts after the domain to filter my interest: example.com/news-opinion vs example.com/science-blog gives me more information that might not be clear from the post title. With a site like Facebook, which hosts so much different value levels of content, it is harder for me to visit facebook.com/233555323 rather than facebook.com/ffmpeg-interest-group/post6665443. For whatever reason, the URL remains something I use to filter my internet consumption. Perhaps it has something to do with the amount of scams or other general reasons not to trust non-human readable URLs, or perhaps it is more due to my distrust of article titles (sensationalism) and seeking a second source to verify my interest. Perhaps I simply bias against Facebook - I don't know. Regardless, I visited the story after thinking about this, and was very surprised to see it was an article by John Carmack - an author I certainly respect. It is a shame for me that both the URL and the title don't reflect that.
- dtf 7y ago
  Another site that comes to mind is the Financial Times, which uses UUIDs for article urls, with no other indication of content. https://www.ft.com/ https://www.ft.com/ https://www.ft.com/content/13be9132-8149-11e9-b592-5fe435b57a3b https://www.ft.com/content/13be9132-8149-11e9-b592-5fe435b57...
  - nikkwong 7y ago
    Oh wow, you really have to hope these are not the canonical links. If so, they are doing themselves an incredible disservice and the fact that no one has brought it to management's attention speaks gravely to the workplace culture there.
    - saurik 7y ago
      Not only does the full URL not show in my browser (Safari), the full URL doesn't even show on this forum (for me it is cut off after "content"). When you receive a URL via chat or on social networks, it is in the form of a preview and might not even show the domain name. The user also isn't typing these URLs in, with or without human readable text in the path. Essentially all of the arguments for making URLs more brittle (which is what you are doing by adding human readable text) are obsolete in a world where URLs have been relegated back to what they were always supposed to be: permanent identifiers to a semantically stable unit of content.
m0dest 7y ago
For all of our sanity, please don't start producing 8-bit H.264 videos with full 0-255 range. If you have enough control over the decoder and renderer to make that hack work, you should just be producing 10-bit HEVC/AV1 video – which does not suffer from these 8-bit banding issues – with a modern color space (Rec 2020, PQ, etc.). And if you don't have choice or control over the decoder and renderer, then it's a good sign that this would be a fragile hack.
- the-dude 7y ago
  Logitech C920 family cameras do exactly that. H264 full range.
- JohnCarmack 7y ago
  My primary concern is for immersive VR media on our Go and Quest platforms, which use Snapdragon 821 and 835 SoC, respectively. The 821 doesn’t support 10 bit on anything, and the 835 only supports it on h265. Software decoding a 4k60 video on mobile isn’t an option, and you can see every bit transition with dark adapted eyes in a vr headset, so getting full 8 bit range is pretty valuable.
  - thrwaycolor 7y ago
    You forget to mention which colorspace you're using. It matters. From Rec. 2020 wikipedia article: Since a larger color space increases the difference between colors an increase of 1-bit per sample is needed for Rec. 2020 to equal or exceed the color precision of Rec. 709. The "correct" solution is to use a modern colorspace (rec 2020) with 10 bits encoding (or 12 bits, if you can). UHD standard mandates at least 10 bits. On the '821 you could gain 1 bit by degrading to an older color space - 709. Full 0-255 range should never be used - they are invalid according to the standard. None of the hardware codecs are validated for 0-255 - you're asking for trouble here. Expect ghosting and weird motion artifacts accumulating from one I-frame to the next I-frame.
    - the-dude 7y ago
      Are you sure about that? Logitech sells H264-encoding cameras which do yuvj420p ( which is full range as I understand it ). YT and other platforms have no problem ingesting these videos. Also the RPi Broadcom chipset does not seem to care.
bArray 7y ago
Version you can view with JS disabled: https://mbasic.facebook.com/permalink.php?story_fbid=2413101932257643&id=100006735798590&_fb_noscript=1 https://mbasic.facebook.com/permalink.php?story_fbid=2413101...
Quarrelsome 7y ago
I still find it kinda odd that this is still the best tool out there for this sort of work. Archaic parameters and options and exceptionally hard to read documentation. Maybe I'm just dumb and not in the domain enough to get a decent handle but everytime I use ffmpeg I feel like a Norse Shaman trying through trial and error to locate the correct orientation of runes for the blessing.
- hazeii 7y ago
  Not sure how true it is, but I read somewhere that the reason it's so good at what it does is because it's used at scale by certain large websites (youtube being specifically mentioned).
  - Quarrelsome 7y ago
    I don't know of any other tool that does what it does. I used it myself when handling video and needing to re-encode or fix up output captured from elsewhere. Its a swiss-army knife of video operations.
  - Daemon404 7y ago
    You've got cause and effect backwards there (mostly; it goes both ways). Disclosure: FFmpeg contributor and Vimeo employee.
- Hello71 7y ago
  there are two problems here. the main problem is that nobody reads the manual, which actually starts out half-decent. the description is very clear, concise, and even contains the oft-whinged-about examples. https://ffmpeg.org/ffmpeg.html https://ffmpeg.org/ffmpeg.html the second problem is that ffmpeg has a lot of features, and nobody wants to go through and fully document hundreds of formats, codecs, and filters.
  - Daemon404 7y ago
    Part of the problem is that the documentation is so vast (the page you link is only one of many, many pages; the documentation has been split out into multiple pages for a while now) that as a user, it's fairly insane to try and consume and comprehend it all, especially if all you want to do is something quick.
Hello71 7y ago
also see "Falsehoods programmers believe about [video stuff]": https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-believe-about-%5Bvideo-stuff%5D.html https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-be.... previously discussed: https://news.ycombinator.com/item?id=13259686 https://news.ycombinator.com/item?id=13259686.
waivek 7y ago
I see a lot of people confused about the host (Facebook). Carmack initially posted a lot of high quality articles on AltDevBlogADay. However, that site went down. Carmack said he moved to Facebook because he feels it's more reliable than other (cleaner) third-party hosts, which might not be around 10 years from now.
- Arkanosis 7y ago
  I've seen a lot of similar posts hosted on Google Plus and shared on HN in the past. They are all gone, now. Sure, Google has a horrible track record when it comes to service durability, but I wouldn't trust Facebook as a reliable host either. Anyone having relied on their @facebook.com email address (2010-2016) may agree.
- cesarb 7y ago
  I just happened to find where he said that: https://news.ycombinator.com/item?id=17068101 https://news.ycombinator.com/item?id=17068101 "Years ago, I felt burned when I wrote several articles for #AltDevBlogADay, and they vanished. I have much more confidence that what I write on FB won't vanish. [...]"
deleted 7y ago
[deleted]

177 points by doppp 7y ago

Interesting find. I have encoded video for over a decade, and I don't think a two month period has gone by where I haven't learned something that convinces me to completely change how or what flags I'm using. This is way beyond that but still.

PorterDuff 7y ago
At the risk of telling you something you know, there's a lot more things to trip on with proper color space conversion than just going from scaled to full range color in 601. It's something of rat's nest to cover all the cases.
- dylan604 7y ago
  After ProRes became a thing, there was a lot of confusion where the metadata saved within the MOV started to include the HD vs SD color primaries. Some programs were able to correctly read this data. Most did not. Most programs did not include the data including Final Cut. The iTunes team started rejecting files exported from FCP because they claimed they were not made correctly and not a valid file. The iTunes team were just reading the spec and built their automation based on that, but not what the real world output from FCP was doing.

VectorLock 7y ago

Its weird to think how John Carmack went from .plan files to Facebook Posts. I think it might be safe to say he had the most widely read .plan file in existence. Wish he would go back to it.

jakear 7y ago
I suspect it’s not his decision to make.
- pvg 7y ago
  You think someone is telling Carmack where he can and can't post? Why one earth would they do that? And why on earth would he put up with it?
  - setpatchaddress 7y ago
    It doesn't have to be coercive. Some of us would like to know what the people who hire us are up to even if we don't personally have any interest in it.
  - jakear 7y ago
    More that the company’s culture says you should post things using the company’s platform, and to not do so might be making a statement he doesn’t want to make. Otherwise, why would he move away from .plan?
    - tqi 7y ago
      Could it be that a lot of people don't know what a .plan file is, or how to read content from one?
      - roghummal 7y ago
        If John Carmack started updating a .plan, people would figure out how to check a .plan.
    - pvg 7y ago
      You said 'it wasn't his decision to make' not 'perhaps he wants to use his employer's platform'. If you don't want to imply it's coercive, you should phrase it in a way that doesn't imply it's coercive.
      - jakear 7y ago
        I believe it’s a “perhaps he feels he should be using his employer’s platform”. A murky coercion middle ground.
kwijibob 7y ago
Yes I wondered this also. It is naturally to use his employers platform. However I wonder if in 10-15 years this post will be lost in Facebook changes, but the .plan files from the 1990s will still be available somewhere.
- enneff 7y ago
  Just as the .plan files were mirrored by some enthusiasts, so too will his Facebook posts.
  - whatshisface 7y ago
    Are they mirrored already? I don't want to visit facebook.
    - hoseja 7y ago
      In this very comment section. https://news.ycombinator.com/item?id=20036710 https://news.ycombinator.com/item?id=20036710
deleted 7y ago
[deleted]
ekianjo 7y ago
Maybe he was asked to publish all his communication on Facebook as part of his career move. Along with a large compensation. Unless he really believes Facebook is the right place to post this kind of things nowadays, who knows?

ajxs 7y ago

Funnily enough, I actually ended up giving myself a crash course in this very topic when encoding frag movies for Quake 3/Quake Live. It was/is common practice to use 'brightskins' on your enemy models in competitive play to make them easier to see, typically being a colour like '#00FF00'. I noticed that when uploading to online media hosting sites like Youtube these bright colours would become noticeably washed out and degraded. My investigation of why this happened led me all the way down this rabbit hole. This was the better half of ten years ago now so I have no idea if it still occurs, but I ended up just rendering the videos using a better brightskin colour that appeared less degraded after conversion... Going through the cycle of 'rerender/reupload/check' to find a replacement colour again and again to experiment sure was a major pain!

zaroth 7y ago

> This is usually visible as ugly banding or blocking in dark scenes. I was just complaining about this a couple weeks ago with some friends after the “Dark” episode of GoT. I was saying the codecs don’t seem to have caught up or they are just being fed at too low a bit rate to really excel at dark scenes. It’s hard to imagine this is just a scaling issue with the color ranges, and if so, how could it possibly not have been fixed?! I have a high end OLED screen in a very dark room, and I was very disappointed by the picture quality. Enough to wish I hadn’t watched it through HBO Go and that I had a better source. I would love to see some before/after scenes showing the effect of the patch.

brigade 7y ago
It's not; full-range YCbCr doesn't exist anywhere in the "pro" or broadcast pipelines that would have been used for GoT. I think he meant to say that the limited dynamic range of 8-bit video manifests most obviously in dark scenes, since that's where standard gamma curves are the most imprecise relative to our vision (which was fine back when TVs couldn't reproduce that range anyway.) The 14% from full-range doesn't help that much; really you need the 4x finer precision you get from 10bit.
brokenmachine 7y ago
>I have a high end OLED screen in a very dark room, and I was very disappointed by the picture quality. Unfortunately for you, high end OLED in a very dark room is literally the most likely conditions under which you'd be annoyed by these problems... :-) HDR is the way to go.
- zaroth 7y ago
  It’s Dolby Vision from the ATV to the set, it’s the source encoding I assume that just couldn’t live up to material. Perhaps bitrate compromises have to be made when you’re streaming to so many people at once, but it’s not usually so glaring.
  - brokenmachine 7y ago

airstrike 7y ago

http://archive.is/K5Fpv http://archive.is/K5Fpv

modriano 7y ago
Thank you. I really didn't want to go on Facebook.
nvartolomei 7y ago
TIL: EE, a mobile provider, blocks archive.is as unsafe content for people under 18. https://twitter.com/nvartolomei/status/1133644305812381696 https://twitter.com/nvartolomei/status/1133644305812381696
- cr3ative 7y ago
  That sounds correct to me, since it could contain NSFW content, and by default, mobile providers in England have content controls switched on.
__HYde 7y ago
http://archive.fo/K5Fpv http://archive.fo/K5Fpv > .IS domain will likely stop working soon[0] [0] https://blog.archive.today/ https://blog.archive.today/

forgotmypw5 7y ago

John Carmack 7 hrs · Adventures with ffmpeg and color ranges. A video legacy issue that can cause a lot of problems is the issue of “limited” versus “full” component range. For arcane reasons related to TV broadcast limitations, many video formats restrict the YUV color components to be in the 16..235 or 16..240 range instead of the full 0..255 range. Losing 14% of the already-barely-enough 8 bit dynamic range is bad enough, but it also often results in the black range starting at a quite visible grey value because most players don’t rescale the range. This is usually visible as ugly banding or blocking in dark scenes. For ffmpeg, the trick to avoid this is to use ‘-pix_fmt yuvj420p’, which says to use the j-for-jpeg full range in a 420 YUV subsampled p-for-planar format. If you are starting with either RGB images, a 10/12 bit format, or a yuvj420p format video as input, then with the libx264 codec, you would get a full range output. Note that any video processing tools used along the way could also limit the range, and once it is gone, there is no getting it back, so you must be very careful and check your entire pipeline! When you us this format, ffmpeg complains about ‘deprecated pixel format used, make sure you did set range correctly’, but you should ignore this warning. Ffmpeg would like the world to move to specifying the range independently from the YUV channel subsampling and layout: Setting ‘-color_range 0 -pix_fmt yuv420p’ makes the output format yuv420p Setting ‘-color_range 1 -pix_fmt yuv420p’ makes the output format yuv420p(tv) Setting ‘-color_range 2 -pix_fmt yuv420p’ makes the output format yuv420p(pc) Unfortunately, this isn’t yet uniformly handled throughout all the internal format tests. The libx265 integration in ffmpeg didn’t support the deprecated yuvj420p pixel format, only the basic yuv420p one, and no matter what I did, my test videos were always coming out limited range. It didn’t matter if you add a ’-color_range 2’, or ‘-x265-params range=full’. Those will change the settings in the VUI (Video Usability Information) section of the output, but the values are still compressed to the limited range. Regardless of the input data, any 8 bit h265 video coming out of ffmpeg was limited range! I walked through the libx265 code looking for range compression, but it turned out that all the damage was being done by ffmpeg before it got to x265. Ffmpeg will automatically convert formats when the output differs from the input, and since x265 only supported yuv420p, any full range input will be processed. Adding ‘-v 48’ to ffmpeg will dump more information, including this auto_scaler invocation, which is what is killing the full color range: [auto_scaler_0 @ 000001f55ecd1a40] w:2048 h:2048 fmt:bgr24 sar:0/1 -> w:2048 h:2048 fmt:yuv420p sar:0/1 flags:0x4 I was about to start hacking the code to at least do what I wanted for my use case, but I tried an appeal to Twitter: https://twitter.com/ID_AA_Carmack/status/1131715388067274753 https://twitter.com/ID_AA_Carmack/status/1131715388067274753 My suspicions were confirmed, but Derek Buitenhuis went ahead and submitted an official patch to get yuvj420p accepted by libx265, and windows builds are already available at https://ffmpeg.zeranoe.com/builds/ https://ffmpeg.zeranoe.com/builds/. I suspect there was probably some way of working around this involving explicit format conversion filters with -src_range and -dst_range overrides, but this is now working as you would expect it: ffmpeg -i source.mp4 -c:v libx265 -pix_fmt yuvj420p dest.mp4 Bravo to ffmpeg!

carmacks_fb 7y ago

The best part about these blog posts is that people discuss the hosting service more than the content of the article at this point. No one is nearly as interested in Carmack's thoughts on codecs as they are about the domain name of the server he's posting to. And yet, we'd all read his stuff no matter where he puts it. But maybe he reaches more outsiders this way? I doubt it.

dearrifling 7y ago

Fun fact: sRGB also has a non-black black point that is (fortunately) virtually ignored by everyone.

devit 7y ago
Source? As far as I can tell in XYZ black is (0,0,0) and the only sRGB value that maps to that is (0,0,0)
- klodolph 7y ago
  This is incorrect. If sRGB (1,1,1) is XYZ (1,1,1), then sRGB (0,0,0) is XYZ (0.0025,0.0025,0.0025). Or if you are using absolute XYZ values, sRGB (0,0,0) is XYZ (0.1901,0.2,0.2178).
klodolph 7y ago
The sRGB black point just means that (0,0,0) is assumed to have a luminance of 0.2 cd/m^2, because your screen reflects ambient light and other factors. Ignoring it will give you nonsensical values when you try to do conversions into other color spaces or when you try to do color comparisons.

slavik81 7y ago

The ffmpeg documentation for YouTube videos[1] suggests using `-pix_fmt yuv420p` with H.264 and there are warnings against using other pixel formats (such as yuv444p) for compatibility reasons. Should the documentation instead be suggesting `-pix_fmt yuvj420p`? What are the risks associated with choosing that output format? [1]: https://trac.ffmpeg.org/wiki/Encode/YouTube https://trac.ffmpeg.org/wiki/Encode/YouTube

_Gyan_ 7y ago
No, limited range is the most common and expected range for YUV videos. Many players don't look or respect the container metadata such as nclx in MP4 or the bitstream VUI flags and assume limited range.
- bscphil 7y ago
  This is correct. In fact, the following statement from the post is nonsense: > For arcane reasons related to TV broadcast limitations, many video formats restrict the YUV color components to be in the 16..235 or 16..240 range instead of the full 0..255 range. Losing 14% of the already-barely-enough 8 bit dynamic range is bad enough, but it also often results in the black range starting at a quite visible grey value because most players don’t rescale the range. This is usually visible as ugly banding or blocking in dark scenes. Of course players "rescale" the range. (It's not exactly rescaling, it's just that a value of 235 IS reference white in Rec. 709.) In fact, the thought if them not doing this is kind of crazy, as Rec. 709 (used in Blurays and elsewhere) as well as Rec. 601 explicitly use limited range YUV. Players that don't use limited range correctly would be out of spec and couldn't play a single Bluray disc correctly. So for the overwhelming majority of use cases, you want regular old yuv420p. The author is picking up on something that actually happens, however. There are Blurays out there that have incorrectly converted YUV values. This happens when for example something is in limited range YUV, and then that data is passed to a converter that treats the data as full range YUV, and compresses it again to fit in limited range YUV. This "doubly compressed" data is embarrassingly common (maybe 5% or so of Blurays have this problem). If you get a Bluray like this, or any video with the same issue, the blacks will be bright grays and the whites will be dull (similar to how the author describes limited range YUV). The player on the end is reading the YUV values correctly, but the data has been doubly compressed so it only undoes one round of this. The issue is well known in the pirate communities as an example. Reliable release groups always fix these broken Blurays in their encodes. This is usually described as "fixing levels" or something like that.

hiisukun 7y ago

Not sure if this kind of meta-comment is appropriate, but I find it difficult to follow a link like this because of the lack of human readable information contained in the URL. If the story was hosted on a personal blog I recognised (like Julia Evans to take a random example), then perhaps the domain would be enough for me to choose to read the article to see if the content is interesting. If the link was to a broader domain that hosts many writers, I might use the parts after the domain to filter my interest: example.com/news-opinion vs example.com/science-blog gives me more information that might not be clear from the post title. With a site like Facebook, which hosts so much different value levels of content, it is harder for me to visit facebook.com/233555323 rather than facebook.com/ffmpeg-interest-group/post6665443. For whatever reason, the URL remains something I use to filter my internet consumption. Perhaps it has something to do with the amount of scams or other general reasons not to trust non-human readable URLs, or perhaps it is more due to my distrust of article titles (sensationalism) and seeking a second source to verify my interest. Perhaps I simply bias against Facebook - I don't know. Regardless, I visited the story after thinking about this, and was very surprised to see it was an article by John Carmack - an author I certainly respect. It is a shame for me that both the URL and the title don't reflect that.

dtf 7y ago
Another site that comes to mind is the Financial Times, which uses UUIDs for article urls, with no other indication of content. https://www.ft.com/ https://www.ft.com/ https://www.ft.com/content/13be9132-8149-11e9-b592-5fe435b57a3b https://www.ft.com/content/13be9132-8149-11e9-b592-5fe435b57...
- nikkwong 7y ago
  Oh wow, you really have to hope these are not the canonical links. If so, they are doing themselves an incredible disservice and the fact that no one has brought it to management's attention speaks gravely to the workplace culture there.
  - saurik 7y ago
    Not only does the full URL not show in my browser (Safari), the full URL doesn't even show on this forum (for me it is cut off after "content"). When you receive a URL via chat or on social networks, it is in the form of a preview and might not even show the domain name. The user also isn't typing these URLs in, with or without human readable text in the path. Essentially all of the arguments for making URLs more brittle (which is what you are doing by adding human readable text) are obsolete in a world where URLs have been relegated back to what they were always supposed to be: permanent identifiers to a semantically stable unit of content.

m0dest 7y ago

For all of our sanity, please don't start producing 8-bit H.264 videos with full 0-255 range. If you have enough control over the decoder and renderer to make that hack work, you should just be producing 10-bit HEVC/AV1 video – which does not suffer from these 8-bit banding issues – with a modern color space (Rec 2020, PQ, etc.). And if you don't have choice or control over the decoder and renderer, then it's a good sign that this would be a fragile hack.

the-dude 7y ago
Logitech C920 family cameras do exactly that. H264 full range.
JohnCarmack 7y ago
My primary concern is for immersive VR media on our Go and Quest platforms, which use Snapdragon 821 and 835 SoC, respectively. The 821 doesn’t support 10 bit on anything, and the 835 only supports it on h265. Software decoding a 4k60 video on mobile isn’t an option, and you can see every bit transition with dark adapted eyes in a vr headset, so getting full 8 bit range is pretty valuable.
- thrwaycolor 7y ago
  You forget to mention which colorspace you're using. It matters. From Rec. 2020 wikipedia article: Since a larger color space increases the difference between colors an increase of 1-bit per sample is needed for Rec. 2020 to equal or exceed the color precision of Rec. 709. The "correct" solution is to use a modern colorspace (rec 2020) with 10 bits encoding (or 12 bits, if you can). UHD standard mandates at least 10 bits. On the '821 you could gain 1 bit by degrading to an older color space - 709. Full 0-255 range should never be used - they are invalid according to the standard. None of the hardware codecs are validated for 0-255 - you're asking for trouble here. Expect ghosting and weird motion artifacts accumulating from one I-frame to the next I-frame.
  - the-dude 7y ago
    Are you sure about that? Logitech sells H264-encoding cameras which do yuvj420p ( which is full range as I understand it ). YT and other platforms have no problem ingesting these videos. Also the RPi Broadcom chipset does not seem to care.

bArray 7y ago

Version you can view with JS disabled: https://mbasic.facebook.com/permalink.php?story_fbid=2413101932257643&id=100006735798590&_fb_noscript=1 https://mbasic.facebook.com/permalink.php?story_fbid=2413101...

Quarrelsome 7y ago

I still find it kinda odd that this is still the best tool out there for this sort of work. Archaic parameters and options and exceptionally hard to read documentation. Maybe I'm just dumb and not in the domain enough to get a decent handle but everytime I use ffmpeg I feel like a Norse Shaman trying through trial and error to locate the correct orientation of runes for the blessing.

hazeii 7y ago
Not sure how true it is, but I read somewhere that the reason it's so good at what it does is because it's used at scale by certain large websites (youtube being specifically mentioned).
- Quarrelsome 7y ago
  I don't know of any other tool that does what it does. I used it myself when handling video and needing to re-encode or fix up output captured from elsewhere. Its a swiss-army knife of video operations.
- Daemon404 7y ago
  You've got cause and effect backwards there (mostly; it goes both ways). Disclosure: FFmpeg contributor and Vimeo employee.
Hello71 7y ago
there are two problems here. the main problem is that nobody reads the manual, which actually starts out half-decent. the description is very clear, concise, and even contains the oft-whinged-about examples. https://ffmpeg.org/ffmpeg.html https://ffmpeg.org/ffmpeg.html the second problem is that ffmpeg has a lot of features, and nobody wants to go through and fully document hundreds of formats, codecs, and filters.
- Daemon404 7y ago
  Part of the problem is that the documentation is so vast (the page you link is only one of many, many pages; the documentation has been split out into multiple pages for a while now) that as a user, it's fairly insane to try and consume and comprehend it all, especially if all you want to do is something quick.

Hello71 7y ago

also see "Falsehoods programmers believe about [video stuff]": https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-believe-about-%5Bvideo-stuff%5D.html https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-be.... previously discussed: https://news.ycombinator.com/item?id=13259686 https://news.ycombinator.com/item?id=13259686.

waivek 7y ago

I see a lot of people confused about the host (Facebook). Carmack initially posted a lot of high quality articles on AltDevBlogADay. However, that site went down. Carmack said he moved to Facebook because he feels it's more reliable than other (cleaner) third-party hosts, which might not be around 10 years from now.

Arkanosis 7y ago
I've seen a lot of similar posts hosted on Google Plus and shared on HN in the past. They are all gone, now. Sure, Google has a horrible track record when it comes to service durability, but I wouldn't trust Facebook as a reliable host either. Anyone having relied on their @facebook.com email address (2010-2016) may agree.
cesarb 7y ago
I just happened to find where he said that: https://news.ycombinator.com/item?id=17068101 https://news.ycombinator.com/item?id=17068101 "Years ago, I felt burned when I wrote several articles for #AltDevBlogADay, and they vanished. I have much more confidence that what I write on FB won't vanish. [...]"

deleted 7y ago

[deleted]