4 ms·
I've gotten curious about FPGAs myself of late, particularly with video capture. A hopefully-on-topic question of my own, if I may: I've seen that some FPGA bo
by exikyut 9y ago
I've gotten curious about FPGAs myself of late, particularly with video capture. A hopefully-on-topic question of my own, if I may:
I've seen that some FPGA boards have HDMI transceivers that will decode TMDS and get the frame data into the FPGA somehow. That got me thinking about various possibilities.
- I want to build a video capture device that will a) accept a number of TMDS (HDMI, DVI, DisplayPort) and VGA signals (say, 8 or 10 or so inputs, 4-5 of each), simultaneously decode all of them to their own framebuffers, and then let me pick the framebuffer to show on a single HDMI output. This would let me build a video switcher that could flip between channels with a) no delay and b) no annoying resyncs and c) because everything's on independent framebuffers I can compensate for resolution differences (eg, a 1280x1024 input on the 1920x1080 output) via eg centering.
- In addition to the above, I also want to build something that can actually _capture_ from the inputs. It's kind of obvious that the only way to be able to do this is via recording to a circular window in some onboard DDR3 or DDR4 (256GB would hold 76 seconds of 4K @ 144fps). My problem is actually _dumping_/saving the data in a fast way so I can capture more input.
I can see two ways to build this
1) a dedicated FPGA board with onboard DDR3, a bunch of SATA controllers and something that implements the equivalent of RAID striping so I can parallelize my disk writes across 5 or 10 SSDs and dump fast enough.
2) A series of FPGA cards, each which handles say 2 or 3 inputs, and which uses PCI bus mastering to write directly into a host system's RAM. That solves the storage problem, and would probably simplify each card. I'd need a fairly beefy base system, though; 4K 144fps is 26GB/s, which is uncomfortably close to PCI-e 3.0 x16's limit of 32GB/s.
I'll admit that this mostly falls under "would be really really awesome to have"; I don't have a commercial purpose for this yet, just to clarify that. That said, my inspiration is capturing pixel-perfect, perfectly-timed output from video cards to identify display and rendering glitches (particularly chronologically-bound stuff, like dropped frames) in software design, so there's probably a market for something like this in UX research somewhere...
- pjc50 9y agoThe capture thing appears to exist: https://www.blackmagicdesign.com/products/hyperdeckshuttle/ https://www.blackmagicdesign.com/products/hyperdeckshuttle/ - presumably it applies lossless compression. Lossless encoding within the h264 container should be possible.
- exikyut 9y agoThat is interesting. Hmm, I just found a teardown video: https://www.youtube.com/watch?v=TLs2CNhbKAs https://www.youtube.com/watch?v=TLs2CNhbKAs I spy a Xilinx Spartan 6 at 2:16!
- mschuster91 9y agoThat Blackmagic stuff is really awesome. Expensive as ..., but worth every penny.
- j_s 9y agoPardon my ignorance, but I thought Blackmagic's claim to fame was always being on the low end of the pricing for their tech.
- j_s 9y agoPerhaps related, if not quite as ambitious: https://hdmi2usb.tv/home/ https://hdmi2usb.tv/home/ - Open video capture hardware + firmware I would also recommend this project if someone doesn't have an idea of their own, and/or doesn't have any fpga experience. So if someone is looking for people with both of those doing work in the open who might be willing to mentor them, finding a way to help this project is a great way to get started.
- exikyut 9y agoThis is quite interesting, thanks! It uses an FPGA with DDR3 incidentally, huh. Thanks very much for this, this is very close to what I'm trying to get at. I agree that figuring out how to help out on this project would be very constructive, I've filed it away.
- jleahy 9y agoThis is a very difficult project. No FPGAs have anything to decode TMDS. What they have is transceivers that will take the multi-gigabit serial signal and give it to you 32-bits or 64-bits at a time. Then it's your job to handle things like bit alignment, 8b10b decoding, etc. You need 6MB of RAM to hold one frame at 1080p, and to do this you'll need to hold one for each input. That requires external RAM (there's no enough on the FPGA), which means you'd need something like 30Gbit/sec of RAM bandwidth, which means you need DDR2 RAM before you've even stored anything (DDR3 is really awkward, I'd stick to DDR2 or LPDDR2, DDR4 is going to be impossible). I'm not even going to comment on the feasibility of implementing SATA.
- exikyut 9y agoI see... ouch. Another comment mentioned the HDMI2USB project, and the main FPGA target of that project uses DDR3 FWIW. I get the impression that managing RAM is just really really really _unbelievably_ hard; not impossible, just pɹɐɥ ʎllɐǝɹ ʎllɐǝɹ.
- DigitalJack 9y agoI don't think it's that bad. Board design is, but if you are working with some standard board, say from Opal Kelly (no affiliation), then you don't have to worry about that. But what about the DRM with HDMI? I have no idea how it works, but I assumed the signal was encrypted.
- orbifold 9y agoCheckout https://git.elphel.com/Elphel/x393 https://git.elphel.com/Elphel/x393, it is a design for an FPGA for a camera, it contains roughly the components you are talking about, on board DDR3, sata output. The code is released under GPL3, so you won't be able to use it for something commercial, but it might be a good starting point to give you an idea how such a design would look like.
- slrz 9y ago> The code is released under GPL3, so you won't be able to use it for something commercial […]. Actually, you can do this just fine. There are no 'non-commercial' clauses or anything like that in the GPL. You just have to release your own sources.