4 ms·

UDP yes. Repeated frames introduce latency. Broadcasters are used to latency measured in lines -- that's microseconds. In the analog days an extra foot on your

UDP yes. Repeated frames introduce latency. Broadcasters are used to latency measured in lines -- that's microseconds. In the analog days an extra foot on your cable would mean the difference between out of sync or not in the chroma domain. We use SDI rather than HDMI, but fundamentally it's a container, with an active picture frame in 4K video of 2160 lines, with 3840 samples per line. Each sample is a 10 bit sample of the luma, and every other sample gets 2 extra 10 bit samples of chroma (The eye being less sensitive to chroma). This is 4:2:2 sampling. This means that a single 4K frame is 2160x3840x20 bits, and in the americas there are about 60 lots of 165mbit, every second. SDI adds various things like blanking thanks to it's legacy of a digitised version of PAL/NTSC. Audio is tiny, <15mbit/sec. Normal HD, with interlacing, is far more reasonable, at 1080x1920x20x30ish, 1.2gbit (less in 25fps countries), but SDI adds another 300mbit a second on top of that (UDP/RTP also adds overhead too). The cheapest SDI routers you can get tend to be blackmagic ones, which are fine. There's a lot of people in the industry that remember the days of analog routers, where they were legitimatly expensive. Since SDI came out thought the electronics is little more than a switch chip. A 40x40 matrix will cost about $1300, but prices increase exponentially. On wan links we tend to use compressed video in the 5-30mbit range (5mbit from somewhere skanky with poor internet). We have forward error correction on that, but the standards aren't great at high bit rates (only coping with 25 lost packets). A 20ms outage on a line with that type of error correction causes on air disruption. Using something like ARQ which allows missing frames to be repeated adds latency, on a link from Europe to Singapore for example it could add 600ms. That's on top of the c. 100ms propagation latency and 400ms encode/decode, pushing the latency from half a second to over a second, and people start getting grumpy. There's still no guarentees either - as if you were to lose 100ms of traffic at 30mbit, you end up peaking your traffic far higher than the 30kbit/ms that you normally send -- to retransmit that 100ms in 100ms, you need to peak at 60mbit. As an example of how important latency is for two-way communication, I'm currently investigating replacing our 12 frame (480ms) latency encoders from a couple of locations in London with ones that run at about 1-2 frames (40-80ms). With wan links sending the same packet on two links doesn't even work. I've seen outages of 20ms on both a private wire and two separate internet paths in the past -- presumably some common router in the subsea cable system does something. Currently using temporal shifting (and some receive buffering) as well. The expectations that the broadcast industry place is somewhat different to the IT industry. 3 second end-to-end latency is nearer 'store and forward' than 'low latency'. Compression in a production facility is unacceptable, and increasingly we're moving away from things like 5:1 compression on wan links and bandwidth becomes cheaper. One broadcaster in the UK has just replaced the main in-country network with an uncompressed network based on multi-path 100gbit wavelengths running over dark fibre. Another has replaced it's core switching network with multi-terabit backplanes. Just getting 10gbit/second in/out through the kernel is a challange - doing it with nanosecond packet timing that the next SMPTE standard currently requires is proving even harder.

exikyut 9y ago
Ah, 10-bit 4K. That completely explains 12Gbps. I'm not entirely sure what you mean by "1920x1080x20x30", particularly the "20". I take the "30" to mean 30fps. SDI rings an "I've heard of that!" bell. Huh, 37MB/s of overheard - nice. :( I get the impression analog routers/multiplexers were basically every input port physically wired to every output port, with a tiny switch to disconnect the lines that weren't being used. Yeah, those would be exponentially expensive, because even though you can't physically have more than one input going to a given output (outside of practical special effects, maybe?), you can't get away from physically wiring all inputs to all outputs. And then I'm guessing the sea of internal wiring was all super high quality and so forth to minimize line losses... 40x40 for $1.3k is _very_ impressive. But yeah, it's just a specialized ethernet switch (albeit with really high maximum internal throughput), so that makes sense. The switch from analog to packet switched routing does make sense financially. Hmm. What did everyone use after everything went digital, but before the switch to packet switching? What video formats got used back then? What time period am I even describing? I just realized I've always been mildly curious. TIL about ARQ, and what to call what I've been using in my own messaging system design ideas. I do wonder though... in the context of video distribution, if you just repeat the last good frame you had if you don't get the next frame in time - why does there need to be realtime acknowledgement at all? Instead of acknowledging every single frame (I assume this is how current systems work), either send periodic beacons containing the last eg 10 or 100 or 1000 last message IDs, eg [ 1, 3, 8, 9, 10, 13, 17, 20, ... ] so the sender compute the number of frames dropped, or even better, if frame IDs are guaranteed to be sequential, the receiver can compute the number of dropped frames and simply just ping back the packet loss percentage value. That would take care of the sender blindly going too fast a la TCP rate control. Then, on the receiving end, if the receiver doesn't get the next frame in time for playback system to show it, it just shows the last received frame until it has a new one. I get the impression ARQ works a bit like Skype on a bad day, where the feed locks up periodically then the image jumps forward in time (if you will) when the next properly decodeable frame comes in - except I suspect ARQ only has very tiny such jitters and jumps. What I just described would probably be equivalent, but (unless I'm horribly mistaken, or misunderstand) with much lower latency. Note - I realize the way things are already done were carefully engineered and there are good reasons behind the techniques currently being used. I presume that the above idea has very possibly been thought of and disregarded - and I'm curious what the reason was. 400ms encode/decode is impressive. 40-80ms even more so. But that sounds perfectly reasonable given that this uncompressed video; I expect the faster encoder simply has faster silicon in it :) TIL about the info about the traffic peaking and retransmission. That was interesting to carefully read through a couple of times :) Talking about encoders and distances, that reminds me - I'm curious about how good those cellular backpack things are, the ones that take umpteen SIMs and seemingly set up a load-balanced aggregate connection over all of them. I'm guessing the latency is somewhere north of "higher than planes fly"? :P What did you mean by "temporal shifting" in the context you used it? I also wonder what the traceroute on the private wire would have looked like. I can't imagine the link was tunneled (haha!) so if nothing showed up that would be perplexing. I remember when satellite links used to have 3 second delays to TV live news broadcasts, only around 2006-2008 or so. Hah. Wow, expectations have (perfectly understandably) moved forward. Agh. I'm 100% sure that it's possible to losslessly compress video and get good speedups... but the realtime requirements really make it so incredibly hard :( I realize HEVC has a lossless profile, but yeah, definitely not realtime. The problem is that anything that compresses at realtime on current silicon (and not anything custom) will probably only shave off a few tens or hundreds of Mbps. :/ TIL about the network upgrades. By "network" here, I presume you mean at least the in-building LAN, but I'm not sure where/what else classifies as "production". Surely the lines to the TV towers aren't uncompressed? I see no need for uncompressed video there (but I may be sorely misinformed). And wow, multi-Tbps backplanes... I was reading about how Facebook had deployed that sort of thing a few months ago, TIL it really isn't that unique. As for getting 10Gbps kernel throughput, I understand a lot of groups are just doing networking in userspace and skipping the kernel IP stack entirely. But 10Gbps at nanosecond timing... haha, when I hear that, I say kick out Linux entirely, because it doesn't offer hard realtime AFAIK. Well, my knowledge may be hugely out of date, but last I heard Linux didn't really like doing realtime "in production". At any rate I know of no projects using it routinely for non-specialist tasks (ie, some kind of everyday desktop-usage use case, not embedded/appliance use-case). I vaguely reading about the SNABB Switch folks doing really really fast packet switching, and they were getting all their speedups by using assembly language and doing instruction/cycle counting. Problem with using Linux is that you have a massive bunch of overhead (the mechanics of C, basically) you can't really shoo out of the way - sure, you can optimize your own stuff in assembly, but what's the point when you have tons of C kernel code darting in and out of the L1/L2/etc caches in between schedules of your process(es)... Hmm. I just realized... it would be so nice if you could tell Linux to give you an entire CPU core. Just don't schedule anything onto this or that CPU, and don't even run kernel code on it. Then you could use the excluded core like a discrete application processor, and run your own kernel on it. THAT would be COOL. But I don't know if you can actually use x86 like that. Somewhat less drastically, I wonder if you could build FPGAs - or, if the chipsets in NICs are amenable and fast enough, just bespoke firmware - and teach the NIC to receive and buffer SDI video frames really, really fast, doing some complex processing in hardware before the kernel gets the data over PCI-e. That reminds of something that may or may not be interesting: https://news.ycombinator.com/item?id=13741975 https://news.ycombinator.com/item?id=13741975 - it's a totally new CPU architecture (it seems that the CPU arch wars/competitions are starting to warm up - nice) that basically, uh, moves the MMU into LLVM, instead of putting it in silicon. It's an interesting idea, but the reason I mention it is because I understand they're already at the point where they're taking orders (and have taped-out silicon to actually deliver). FWIW I was i336_ in that thread (accidentally locked the account, heh) but apart from the interesting interaction I have no affiliation.

exikyut 9y ago

Ah, 10-bit 4K. That completely explains 12Gbps. I'm not entirely sure what you mean by "1920x1080x20x30", particularly the "20". I take the "30" to mean 30fps. SDI rings an "I've heard of that!" bell. Huh, 37MB/s of overheard - nice. :( I get the impression analog routers/multiplexers were basically every input port physically wired to every output port, with a tiny switch to disconnect the lines that weren't being used. Yeah, those would be exponentially expensive, because even though you can't physically have more than one input going to a given output (outside of practical special effects, maybe?), you can't get away from physically wiring all inputs to all outputs. And then I'm guessing the sea of internal wiring was all super high quality and so forth to minimize line losses... 40x40 for $1.3k is _very_ impressive. But yeah, it's just a specialized ethernet switch (albeit with really high maximum internal throughput), so that makes sense. The switch from analog to packet switched routing does make sense financially. Hmm. What did everyone use after everything went digital, but before the switch to packet switching? What video formats got used back then? What time period am I even describing? I just realized I've always been mildly curious. TIL about ARQ, and what to call what I've been using in my own messaging system design ideas. I do wonder though... in the context of video distribution, if you just repeat the last good frame you had if you don't get the next frame in time - why does there need to be realtime acknowledgement at all? Instead of acknowledging every single frame (I assume this is how current systems work), either send periodic beacons containing the last eg 10 or 100 or 1000 last message IDs, eg [ 1, 3, 8, 9, 10, 13, 17, 20, ... ] so the sender compute the number of frames dropped, or even better, if frame IDs are guaranteed to be sequential, the receiver can compute the number of dropped frames and simply just ping back the packet loss percentage value. That would take care of the sender blindly going too fast a la TCP rate control. Then, on the receiving end, if the receiver doesn't get the next frame in time for playback system to show it, it just shows the last received frame until it has a new one. I get the impression ARQ works a bit like Skype on a bad day, where the feed locks up periodically then the image jumps forward in time (if you will) when the next properly decodeable frame comes in - except I suspect ARQ only has very tiny such jitters and jumps. What I just described would probably be equivalent, but (unless I'm horribly mistaken, or misunderstand) with much lower latency. Note - I realize the way things are already done were carefully engineered and there are good reasons behind the techniques currently being used. I presume that the above idea has very possibly been thought of and disregarded - and I'm curious what the reason was. 400ms encode/decode is impressive. 40-80ms even more so. But that sounds perfectly reasonable given that this uncompressed video; I expect the faster encoder simply has faster silicon in it :) TIL about the info about the traffic peaking and retransmission. That was interesting to carefully read through a couple of times :) Talking about encoders and distances, that reminds me - I'm curious about how good those cellular backpack things are, the ones that take umpteen SIMs and seemingly set up a load-balanced aggregate connection over all of them. I'm guessing the latency is somewhere north of "higher than planes fly"? :P What did you mean by "temporal shifting" in the context you used it? I also wonder what the traceroute on the private wire would have looked like. I can't imagine the link was tunneled (haha!) so if nothing showed up that would be perplexing. I remember when satellite links used to have 3 second delays to TV live news broadcasts, only around 2006-2008 or so. Hah. Wow, expectations have (perfectly understandably) moved forward. Agh. I'm 100% sure that it's possible to losslessly compress video and get good speedups... but the realtime requirements really make it so incredibly hard :( I realize HEVC has a lossless profile, but yeah, definitely not realtime. The problem is that anything that compresses at realtime on current silicon (and not anything custom) will probably only shave off a few tens or hundreds of Mbps. :/ TIL about the network upgrades. By "network" here, I presume you mean at least the in-building LAN, but I'm not sure where/what else classifies as "production". Surely the lines to the TV towers aren't uncompressed? I see no need for uncompressed video there (but I may be sorely misinformed). And wow, multi-Tbps backplanes... I was reading about how Facebook had deployed that sort of thing a few months ago, TIL it really isn't that unique. As for getting 10Gbps kernel throughput, I understand a lot of groups are just doing networking in userspace and skipping the kernel IP stack entirely. But 10Gbps at nanosecond timing... haha, when I hear that, I say kick out Linux entirely, because it doesn't offer hard realtime AFAIK. Well, my knowledge may be hugely out of date, but last I heard Linux didn't really like doing realtime "in production". At any rate I know of no projects using it routinely for non-specialist tasks (ie, some kind of everyday desktop-usage use case, not embedded/appliance use-case). I vaguely reading about the SNABB Switch folks doing really really fast packet switching, and they were getting all their speedups by using assembly language and doing instruction/cycle counting. Problem with using Linux is that you have a massive bunch of overhead (the mechanics of C, basically) you can't really shoo out of the way - sure, you can optimize your own stuff in assembly, but what's the point when you have tons of C kernel code darting in and out of the L1/L2/etc caches in between schedules of your process(es)... Hmm. I just realized... it would be so nice if you could tell Linux to give you an entire CPU core. Just don't schedule anything onto this or that CPU, and don't even run kernel code on it. Then you could use the excluded core like a discrete application processor, and run your own kernel on it. THAT would be COOL. But I don't know if you can actually use x86 like that. Somewhat less drastically, I wonder if you could build FPGAs - or, if the chipsets in NICs are amenable and fast enough, just bespoke firmware - and teach the NIC to receive and buffer SDI video frames really, really fast, doing some complex processing in hardware before the kernel gets the data over PCI-e. That reminds of something that may or may not be interesting: https://news.ycombinator.com/item?id=13741975 https://news.ycombinator.com/item?id=13741975 - it's a totally new CPU architecture (it seems that the CPU arch wars/competitions are starting to warm up - nice) that basically, uh, moves the MMU into LLVM, instead of putting it in silicon. It's an interesting idea, but the reason I mention it is because I understand they're already at the point where they're taking orders (and have taped-out silicon to actually deliver). FWIW I was i336_ in that thread (accidentally locked the account, heh) but apart from the interesting interaction I have no affiliation.