6 ms·
Video codecs and 4k
- tatersolid 4y agoSort of a Wikipedia-style overview of video codec history. If you were expecting some new info or insight, skip the article.
- Aardwolf 4y agoThat 320x240 need for speed intro video doesn't look bad at all to me, for having to be rendered on a 486 era computer! The low resolution pixels are visible, but I don't see many blocking / MPEG artefacts. I've seen much worse 240p videos.
- aidenn0 4y agoThe low frame rate is visible to me, as is the occasional color banding. The spatial resolution looks at least as good as a VHS recording of broadcast TV.
- rasz 4y agohttps://wiki.multimedia.cx/index.php/Electronic_Arts_TGV https://wiki.multimedia.cx/index.php/Electronic_Arts_TGV loss-less intra frame compression, and block-based motion and VQ for inter frames with ADPCM audio. Pretty neat for something running fine on 486DX2/66
- ogurechny 4y agoNFS (on both consoles and PC) didn't have any MPEG-encoded videos. See https://wiki.multimedia.cx/index.php/Cinepak https://wiki.multimedia.cx/index.php/Cinepak https://wiki.multimedia.cx/index.php/Electronic_Arts_TGV https://wiki.multimedia.cx/index.php/Electronic_Arts_TGV for 3DO and PC video formats, respectively. You can get the original files (ATTRACT[1-3]) from the first link (ffmpeg/mplayer had a decoder for that) or the reencodes someone made from the second link. https://ia803209.us.archive.org/view_archive.php?archive=/29/items/need-for-speed-se/Need%20for%20Speed%20SE.iso https://ia803209.us.archive.org/view_archive.php?archive=/29... https://archive.org/details/nfsse-videos https://archive.org/details/nfsse-videos Oh, and the video itself seems to be from a PlayStation version, judging by the graphics in the end. Then the title should be “1996”, and the hardware decoder for more advanced MotionJPEG-like video could be used. Edit: Sorry, that's NFS1 Special Edition intro (without Road&Track co-branding). Before talking about bus bandwidths and video card types, author had to mention that you simply couldn't decode such a complex format on consumer devices at that time, their CPUs weren't fast enough. Back then people used dedicated players with options to trade quality for speed to decode just the MP3 audio files. For video, developers relied on simple codecs with integer math, line/block/area/frame copy/movement/discard operations, and reduced bpp data, like Cinepak, Indeo, and many custom ones. Their metric wasn't the quality level, but amount of frame data that could be updated by decoding CPU in a given time. Everything that couldn't fit got dropped. Some of those allowed to decode at arbitrary locations on arbitrary canvas sizes, and implement screen/region scrolling/updating through the same decoder, if needed. Game designers could cheat by sticking to highly compressible images. Shots with limited color palette and static (perfectly static) CG backgrounds looked better. For example, all of Cyberia FMV sequences only have 15 fps, but most of the time you don't notice that.
- ksec 4y ago>But even MPEG-1 supports a maximum resolution of 4095×4095, and a maximum bandwidth of 100 mbit. So it is technically possible to encode 4k (3840×2160) content even in MPEG-1, at decent quality. The reality is that that despite MPEG-1 and MPEG-2 support those resolution, none of the mainstream implementation supports it. ( Although you could always use Software implementation as MPEG-2 is now patent free ) Hence when you go 4K, only newer version of MPEGs are used. It is also missing MPEG-5 with EVC. And how not only MPEG-4 has different codec, AVC itself introduces many different profiles where compression efficiency differs greatly.
- malkuth23 4y agoTrue. I work with media server playback stuff for live events etc. Back in the day, we only used mpeg2 constant bitrate codec because it allowed us to sync the playback across multiple systems. The first 4k videos we could play were just a slightly hacked version of mpeg2. The software I used adopted the extension .mxl and claimed they sort of invented a new thing, but truthfully, it was just an mpeg2 encoder/decoder that ignored that 4095 rule. Now we use HAP family (DXT/S3 based) or NotchLC (which is a specialized codec for media servers that is fairly encumbered with patents).
- rasz 4y ago>Now we use HAP family (DXT/S3 based) You mean Apple Road Pizza, right? :) https://en.wikipedia.org/wiki/Apple_Video#Four_color_mode https://en.wikipedia.org/wiki/Apple_Video#Four_color_mode
- giantrobot 4y agoMPEG-4 part 2 and h.263 also support multiple profiles. MPEG-4 part 2 is based off of h.263 but with basically all the features turned on with its own additional features available. Annoying the author doesn't seem to understand that multiple standards bodies exist. The h.xxx codecs are specified by the ITU. MPEG is a totally different organization. The MPEG-x standards are specific video encoding and delivery standards. The ITU also has delivery specifications but those are mostly about telecommunications delivery.
- 4y ago
- seanalltogether 4y agoVideo deinterlacing is one of those topics that I always felt would be a good way to teach myself deep learning techniques. The idea that you have a previous frame and a next frame with information to help you fill in the gaps of the current frame is really intuitive and easy to visualize. I'm sure this has been solved really well by now, but it seems like a good problem to learn with.
- IshKebab 4y agoDefinitely already been done, but yeah it's pretty much ideal for ML.
- NonNefarious 4y agoI wouldn't say it has been solved all that well. The problem is the temporal difference between the two fields, so moving objects have a "comb" distortion while more static parts of the image mostly don't. Interlacing should never have been a part of a digital-TV standard, but broadcasters pushed a bunch of FUD and lies during the progressive-vs.-interlaced debate. They whined that going progressive "would break their pipeline," when that's obviously BS because they'd been playing film-originated material for decades. Acquisition should always have taken place in a progressive format, even if transmission continued to be interlaced for a while. Now we're stuck with a bunch of historical footage that looks like shit. And then there's the asinine perpetuation of non-integer frame rates; those too should have been abolished with digital TV.
- Melatonic 4y agoYea seriously - interlacing made sense only for when it was actually required by the technology. We should have dropped it at the first chance we got
- babypuncher 4y agoInterlacing should never have been part of any digital video standard, in my opinion. All it has done is cause problems. Early digital productions shot in 720p look better than their 1080i peers.
- HidyBush 4y agoI expected a treatise about how different video codecs perform when dealing with 4k video, not a very good title imho
- NonNefarious 4y agoNor an entirely accurate article.
- klodolph 4y ago> If we want to get a bit retro here, convincing video playback on a consumer PC more or less started when hardware cards such as the Video Blaster arrived on the market. A bit odd to give QuickTime a snub here, which was released in 1991.
- rasz 4y agoAuthor confused hardware and assumed barebones single frame framegrabber was a video accelerator. QuickTime Road Pizza https://en.wikipedia.org/wiki/Apple_Video https://en.wikipedia.org/wiki/Apple_Video was one of the first viable universal video codecs on the desktop. Its not a coincidence Premiere started out on Mac https://en.wikipedia.org/wiki/Adobe_Premiere#History https://en.wikipedia.org/wiki/Adobe_Premiere#History
- phoboslab 4y ago> MPEG is not free. A license is required for using MPEG. For MPEG-1 this is not true anymore. All patents have expired. It's also far less computationally expensive and easier to implement/integrate than more modern codecs. This makes MPEG-1 a "good enough" solution for (e.g.) short video clips in a game. Disclaimer: I wrote pl_mpeg[1], so my opinion is biased. [1] https://github.com/phoboslab/pl_mpeg https://github.com/phoboslab/pl_mpeg
- corysama 4y agoFun fact: The PlayStation2 (and maybe PS1?) had an "MDEC" coprocessor that was basically a JPEG decoder chip. So, in one game we used low-rez MJPEG streams for the characters' pop-up, talking-head, conversational clips. The were small enough to keep in the 32MB of main RAM for zero-latency and free to decode any frame on the fly as needed.
- pvg 4y agoand maybe PS1? Yeah it started on the PS1, as you say: https://news.ycombinator.com/item?id=27699899 https://news.ycombinator.com/item?id=27699899
- corysama 4y agolol. Been a long time since I worked on a PS1 ;)
- Melatonic 4y agoMotion JPEG was once the bain of my existence - the very first DSLR's to get video recorded in mjpeg (it was an afterthought tacked on - they had no idea it would revolutionise an industry). Many of the free decoders were quite crap and did not do a good job when you were trying to squeeze every last bit of detail out of a 720P mjpeg stream - there was one significantly better one which I am forgetting the name of now. It was paid but I got a hold of it eventually somehow (as a broke college student) through a friend or class. One plus of it however from what I remember is that the format would play on even the crappiest hardware performance wise
- rasz 4y ago1. As GL1zdA already pointed out in blog comments 1992 Creative Video Blaster CT6000 was a single frame screen grabber and not a video encoding/decoding accelerator. YT "Creative Videoblaster CT6000 in action" https://www.youtube.com/watch?v=5LrwSGRflPE https://www.youtube.com/watch?v=5LrwSGRflPE shows what it does. http://web.archive.org/web/20000829095056/http://www.ctlsg.creaf.com/support/specs/video/ct6000.html http://web.archive.org/web/20000829095056/http://www.ctlsg.c... "converts a standard full-motion video image (composite video source) into a format for display on a computer graphics monitor" emphasis on the _image_. "It can position and scale the video image onto the output display and allows the video image to be merged with computer graphics for interactive multimedia applicaiton." means overlay analog video on VGA monitor. The first true video decoding accelerator on PC market came out of David Sarnoff Research Center owned by RCA, later purchased by GE, then finally video tech carved out and sold to Intel. Called Digital Video Interactive https://en.wikipedia.org/wiki/Digital_Video_Interactive https://en.wikipedia.org/wiki/Digital_Video_Interactive it did in fact "real time audio and video decompression, 30 frames per second with a resolution of 320x240". Intel repackaged it into one chip i750 and called it Indeo. Creative released a product based on it in mid 1994 - Video Blaster RT300. PCI bus was already mainstream in 1994, as were Pentiums. You most likely blurred those two together into one product. 2. "so the quality could be very high. Streams of 8 to 12 mbit of data for a single channel were no exception." SD video DVD bitrate is 10Mbit, and here we have OTA stations pushing 720p/1080i at same or lower bw using same codec (US mpeg2). Whole multiplexer in ATSC offers ~20Mbit, and then TV stations cram 3-4 signals in there. Lu Goughs "DRW2022 – Pt5: Digital TV (DVB-T), Digital Radio (DAB+) & FM Radio Analysis" https://goughlui.com/2022/06/05/drw2022-pt5-digital-tv-dvb-t-digital-radio-dab-fm-radio-analysis/ https://goughlui.com/2022/06/05/drw2022-pt5-digital-tv-dvb-t... blog post offers good overview of the OTA TV broadcasting shitshow as seen from Australian perspective. 3-4Mbit is the average. There is nothing "very high quality" about OTA HD TV broadcast. "‘gold standard’ of 4k 60 fps video, streaming services is where you’ll find it" you mean Netflix 4K 24 fps at 15Mbit? :-) That is the reality we are living in now. Apple is an outlier doing 25-40Mbit, everything else is bad. 3 "CRT has some afterglow after the ray has scanned a given area, the even field was still visible somewhat as the odd field was drawn. So the two fields would blend somewhat together" This is simply not how TV works, there is no somewhat, there is no afterglow. CRT goes dark after about 10 lines have been scanned in. "An LCD screen does not have the same ‘afterglow’ that a CRT has" LCD does in fact retain whole picture until the next update, aka "full afterglow" as you would put it. "so there’s no natural ‘deinterlacing’ effect that blends the screens together." The natural deinterlacing effect happens in viewers brain, not on the Screen. You got this entirely backwards. The whole reason interlaced picture doesnt work on LCD is because previous frame stays on the screen. I highly recommend "How a TV Works in Slow Motion - The Slow Mo Guys" https://www.youtube.com/watch?v=3BJU2drrtCM https://www.youtube.com/watch?v=3BJU2drrtCM 4. "HD, another new standard was required, as DVD was limited to SD." Sony used HD as an excuse to push new format with new licensing fees. Nothing, other than corporate greed, necessitated new storage medium. h.264 was already on the market, as were other efficient codecs. There were even failed attempts like Versatile Multilayer Disc using standard DVD with VC-1 coding. 5. 4K "special kind of confusion" who? This is the first time I hear about people associating 4K with new video codecs.
- PaulHoule 4y agoOTA broadcasters have to upgrade to ATSC 3 to support 4K video. You either need to get a new TV set or an ATSC 3 decoder to watch tune it in. In Syracuse (about 50 miles away from me) the three network broadcasters collaborate to run one ATSC 3 transmitter with big which I can tune in with https://www.silicondust.com/product/hdhomerun-flex-4k/ https://www.silicondust.com/product/hdhomerun-flex-4k/ The picture on ATSC 3 is high-quality 1080i/720p with an aggressive surround soundtrack which sounds great for sports events on my home theater upstairs. I wind up watching the ATSC 1 channels more because: ATSC 1 signals have subchannels like Comet (I am a sucker for Quantum Leap, Baby Godzilla, Time Bandits, ...), the new audio codec in ATSC 3 doesn't work on every device, and the ATSC 3 signal is a little less robust than most of the ATSC 1 signals. I think Sinclair broadcasting (the owner of the transmitter) is more interested in embedding sports betting into sports events as an interactive application than they are in 4K. I think that will wait until they start turning off ATSC 1 transmitters and turning on more ATSC 3 transmitters, but that will take a long time.
- ipython 4y agoI looked into this recently when I moved houses and was without cable for a few weeks. I thought about purchasing an ATSC 3 box to try it out instead of just using an antenna with my TVs internal ATSC 1 decoder. I will never adopt ATSC 3 because they have built targeted advertising and viewer tracking into the spec. See for example https://rampedup.us/atsc-3-0-marketing-opportunity/ https://rampedup.us/atsc-3-0-marketing-opportunity/. It’s really too bad but to me this is the final nail in the coffin for ota programming in my opinion.
- Melatonic 4y agoWell that is extremely disappointing. Sinclair as a media company is also shady as fuck so I am not surprised they are the ones going in on this. Maybe this will create a black market for ATSC 3 boxes that have been hacked to remove / block all tracking features - sounds like a fun project for someone here :-D
- PaulHoule 4y ago
- lostmsu 4y agoIf anyone's interested in the programmatic side, I found the new Windows APIs for encoding to be very sufficient for both H.264 and HEVC: https://docs.microsoft.com/en-us/uwp/api/Windows.Media.Transcoding.MediaTranscoder https://docs.microsoft.com/en-us/uwp/api/Windows.Media.Trans... I does not have as many tunable options as ffmpeg, but served me better for the purpose of multimonitor screen capture (~ 7k by 3k total resolution) precisely because it flings DXGI surfaces directly without the need to copy pictures from GPU to RAM and back for encoding.
- _gabe_ 4y agoThis article was amazing! I struggled through learning to use ffmepg as a C++ library, and then I found out how restrictive their licensing was and about mpeg patent laws which was a huge bummer. I'm building a video editor for mathematical animations, it's open source and I wanted to distribute a single executable as a paid option if you don't want to compile it yourself. Unfortunately this is borderline impossible with ffmpegs restrictive licensing. It looks like AV1 will be a great replacement, and seems like it will be widely supported in the future. The BSD-3 license also seems way more permissive, which is a huge plus. Does anyone have any experience using it as a library? I'll definitely be experimenting with it, but I wanted to see if anyone else had a good experience with it as opposed to ffmpeg or something :) Edit: I just wanted to add that my mis-capitalization of ffmpeg is completely intentional ;)
- kazinator 4y agoCalling the cycle "endless" is an exaggeration; I don't see any push anywhere for 16K video at 300 FPS.