10 ms·
I haven't pulled the patents, but they probably have to do with compressing the pre-debayered image. Apple sends Red a big check every year for the same patents
by ryandamm 4y ago
I haven't pulled the patents, but they probably have to do with compressing the pre-debayered image. Apple sends Red a big check every year for the same patents; I believe they're part of ProRes but not sure.
Essentially, by compressing the raw RGB data coming off the sensor (where each pixel only has R, or G, or B) you avoid the data explosion of interpolating color values at each pixel. It's a technique for compressing raw files, and Red was early to the game.
Perhaps someone on here has more detailed technical info, I haven't actually read the patents, just talked to engineers close to the project.
- NonNefarious 4y agoThis is really a clinic on the dereliction of the USPTO and erroneous court rulings that have led to software patents. But... the fact is that Red ripped off JPEG2000 and are now suing over it. And why not, when they're abetted by a corrupt patent regime.
- acchow 4y agoMindblowing that this technique is patentable
- int0x2e 4y agoUntil tested in court, many, many things are patentable...
- JacobThreeThree 4y agoIt's been tested in court. https://www.engadget.com/2019-11-11-apple-prores-raw-red-patent-dispute.html https://www.engadget.com/2019-11-11-apple-prores-raw-red-pat... Major patent reform is required.
- ryandamm 4y agoWe actually have had major patent reform! Look into IPR review, the Alice Supreme Court decision, and the patent reform act of ... was it 2012? 2013? The IP landscape is wildly different than it was when these patents were first filed. That they've held up under scrutiny implies they're not _total_ junk. Also, it probably still needs reform. But my understanding from talking to lawyers (who I was paying, to be fair), it's better than it was.
- rasz 4y agoApple is really good at losing patent cases in court, even when someone sues using a patent copied verbatim from Apple itself (S3TC - road pizza video codec)
- ryandamm 4y agoKeep in mind that video cameras of that era were mostly not bayer pattern sensors, at least for professional television cameras. They were three, full raster sensors, typically with an interference-based prism block shunting blue, green, and red light to different sensors. These were so-called "three chip" cameras, the optics of which dates back to analog cameras that used tubes. Bayer pattern sensors were more common in DSLRs, because they were an extension of 35mm film, and you had to have a single, flat imaging device to work with film optics. In the film industry, the only digital cameras were the Sony F900 series (iirc) which I believe had the three-chip design, the Thomson Viper (again, memory foggy), and maybe the Panasonic Varicam... I don't know if the Viper or Varicam were single sensor, but they evidently weren't Bayer pattern or didn't compress the Bayer pattern. (I think at least one had RGB stripe layout, which is just inherently inferior.) So yeah, it's obvious now. But it was patentable then, because nobody (or very few) had ever made a Bayer pattern sensor for a video camera. It's easy to forget how long ago 2007 was, at least until I try to get out of bed in the mornings.
- NonNefarious 4y agoThere were Bayer sensors before Red. If I remember correctly, three-chip cameras became impractical at higher resolutions because of the focusing problems that resulted from refraction. Red, green, and blue wavelengths refract at different rates... which was a problem when they're all coming through the same lens. I don't remember details beyond that.
- ryandamm 4y agoI think it had more to do with the requirement that three chip camera lenses be telecentric (see my earlier comment). Tends to make worse lenses that are more expensive.
- NegativeLatency 4y agoThere's a good wikipedia page for anyone interested in how the 3 sensors were implemented: https://en.wikipedia.org/wiki/Three-CCD_camera https://en.wikipedia.org/wiki/Three-CCD_camera
- teawrecks 4y agoSeriously. I zip my raw files for storage on my NAS. Would that be an infringement?
- gjsman-1000 4y agoApple actually tried (and failed) to break RAW from the RED royalties in 2019, so I believe you are correct. https://www.engadget.com/2019-11-11-apple-prores-raw-red-patent-dispute.html https://www.engadget.com/2019-11-11-apple-prores-raw-red-pat...
- amelius 4y ago> Essentially, by compressing the raw RGB data coming off the sensor (where each pixel only has R, or G, or B) you avoid the data explosion of interpolating color values at each pixel. It's a technique for compressing raw files, and Red was early to the game. So the novelty is that they separate R,G and B planes before compression? There are other compression algorithms that work with other colorspaces, giving even better compression. E.g. splitting luminance and chromatic components. I think these algorithms might also be older.
- chongli 4y agoDigital camera image sensors only have one image plane. Each pixel is recorded as a grayscale value where the light has passed through a red, green, or blue filter. To get a complete pixel (with values for all 3 colours) requires software to interpolate a pixel with its neighbours.
- ryandamm 4y agoWell... yeah, that's the crux of it. But the patented part is treating each channel separately; the actual compression algorithm isn't at issue (which, in Red's case, is JPEG2000). That said, as soon as you get into colorspace conversions or color sampling conversions (like YUV, etc), you're baking in choices and dealing with quantization and roundoff errors. If you want the most pristine image possible for dealing with it in post, you want the raw photosite information. And if you want that, without deBayering (which introduces those same errors!), you get to pay Red a royalty check. I don't make the rules, I'm just pointing out what the stakes are here. Yeah, it's pretty fundamental and obvious now, but at the time no one was selling a professional video camera with a single chip for that price. It was a different world.
- car_analogy 4y ago> So the novelty is that they separate R,G and B planes before compression? > Yeah, it's pretty fundamental and obvious now, but at the time no one was selling a professional video camera with a single chip for that price. It was a different world. I don't see how no such cameras being on the market makes the software any less obvious, unless we are totally abdicating the meaning of "obvious" to reduce it to "non-novel". But if that were the law's intent, it wouldn't require non-obviousness and novelty - just one or the other would suffice.
- chongli 4y agoFor those who may not be aware: digital cameras capture a grayscale image with a single value per pixel. The image sensor is covered by an array of colour filters (red, green, blue) in a pattern called the Bayer pattern. Typically, one row of filters will alternate RGRGRGRGRG and the next row will alternate GBGBGBGBGB. "Debayering" involves interpolating the colour-filtered grayscale values of neighbouring pixels into each pixel, and then often applying some software filtering to remove dead pixels, noise, and sharpen the image.
- bruce343434 4y agoDoesn't that mean green is over represented per area? Why not something like RGBRGBRGB GBRGBRGBR BRGBRGBRG
- mafuy 4y agoThe human eye is particularly sensitive to green. This is also why the 5:6:5 encoding of RGB has its focus on green.
- fomine3 4y agoAlso why Pentile OLEDs exist
- verall 4y agoGreen is in the middle of the spectrum so it is the best for detecting amount of light. Color is determined by the G-R and G-B ratios. There are sensors with more esoteric Bayer filters, like to better capture in low light there are sensors which replace one of the G's with an N (neutral light / "no" color filter)
- jeffbee 4y agoAren't there still plenty of cameras using 3 separate sensors and an optical beam splitter, instead of a Bayer filter array?
- 4y ago
- innocenat 4y agoApple is paying RED for #9,230,299. A quick look at the claim doesn't seem to have this patent listed.
- blenderdt 4y agoI believe there are two things to keep in mind here: 1. There is the idea of compressing each Bayer color. 2. There is the compression method. I believe this issue is about point 2, the compression. But it seems that Nikon is using a different compression so I am not sure Red will win this. Edit: from the legal document, I believe it is about this part: "The camera can be configured to transform blue and red image data in a manner that enhances the compressibility of the data. The data can then be compressed and stored in this form. This allows a user to reconstruct the red and blue data to obtain the original raw data for a modified version of the original raw data that is visually lossless when demosaiced. Additionally, the data can be processed so the green image elements are demosaiced first and then the red and blue elements are reconstructed based on values of the demosaiced green image elements."
- AnonCoward4 4y agoThat quote sounds a lot like 1. (the idea of compressing each Bayer color), but I might misunderstand it.
- akira2501 4y agoSounds a lot like converting "left-right" audio into "mid-side" audio for the same benefits.
- rasz 4y ago2 is RED obfuscating ordinary JPEG2000
- blenderdt 4y agoEdit 2: Nikon is using the TicoRAW compression from intoPIX. So I wonder why Red isn't suing intoPIX.
- Strom 4y agoI have a guess, but I haven't actually researched this specific case to be sure. intoPIX seems to be based in Belgium. I'm not sure exactly what type of patents RED has, but there are types like software patents which don't exist in the EU. Or perhaps it isn't about the type and RED just straight up doesn't have EU patents even if they theoretically could. Combine that with intoPIX (and I haven't verified this) only operating inside the EU, or at least outside of the US. You end up in a situation where RED would have difficulty suing intoPIX, because intoPIX doesn't operate in a jurisdiction where RED holds the relevant patents. Once Nikon enters the picture, things change. Nikon clearly has US presence. Of course it could also just be that RED doesn't care about small fish and only started caring now because Nikon has the potential for a lot of revenue.
- walrus01 4y agofor people accustomed to looking at highly compressed video data (like mpeg2, H264 or H265/HEVC file, VP9, AV1, etc), what we're looking at here is a whole other category of barely-compressed high bitrate video used by professional cameras. It saves a lot of SSD recording storage space vs truly raw video but also leaves the video in a state to be further edited and cut together, then compressed into a final output product much later in the work flow. The ordinary consumer will never see these video file formats, unless they go buy at least a $2500 blackmagic pocket 6k pro and add another $2500 of lenses and accessories to make it useful.
- newsclues 4y agohttps://patents.justia.com/inventor/thomas-graeme-nattress https://patents.justia.com/inventor/thomas-graeme-nattress