5 ms·

3K, 60fps, 130ms: achieving it with Rust

38 points by lukastyrychtr 6y ago

snvzz 6y ago
130ms is a world better than 500ms and a much welcome improvement, but it is still terrible. Latency happens throughout the whole stack; Unfortunately much would need to be fixed outside this project to achieve any further significant improvement. Operating System, firmware, blackbox hardware are some other non-negligible sources of latency. Everything adds up.
- sgillen 6y ago
  Is it? They claim to be state of the art in this space. They are processing(?) and sending 3k video over the network, I think 130ms sounds pretty good.
  - snvzz 6y ago
    130ms means 8 whole frames behind at 60fps. You'll have to add the latency of the network on top of that. My point isn't to dismiss their work (any work on reducing latency is very welcome), but rather to highlight that 130ms is still bad, and that the rest of the stack (OS, firmware, blackbox hardware) is definitely a problem.
- dan-robertson 6y ago
  The post claims they achieved that by controlling the whole stack
  - snvzz 6y ago
    I saw that, but I still saw their "capture drivers" configuration example. As I understand it, they have little in terms of outside dependencies, but still do run their software on a common OS. Unlike seL4[0], Linux/OSX/Windows are unsuitable for real time. [0]: https://sel4.systems/About/seL4-whitepaper.pdf https://sel4.systems/About/seL4-whitepaper.pdf
- Carpetsmoker 6y ago
  What would a "non-pathetic" latency look like? What improvements in "the stack" would be needed? The 130ms is "the time from light hitting the camera to when it appears on-screen on the other side". The article doesn't really specify where the latency happens, but with just the encoding/decoding of video alone I think you'll very quickly get latencies in roughly the ~100ms ballpark no matter what.
  - ponker 6y ago
    Not “no matter what,” but perhaps “running on a multitasking operating system.” With a hardware encoder and 1-frame buffer you could get down to 50ms.
    - vertex-four 6y ago
      Across the internet, at all times?
      - ponker 6y ago
        No, just exclusive of transport. So 50ms in-the-building and another 50ms for SF-NY and another 50ms for SF->Moscow or similar.
    - snvzz 6y ago
      As anyone that ever used AmigaOS would tell you, multitasking is not an excuse for poor latency behavior. seL4[0] will guarantee (with formal proof) a low maximum latency despite multitasking. [0]: https://sel4.systems/About/seL4-whitepaper.pdf https://sel4.systems/About/seL4-whitepaper.pdf
  - Kuinox 6y ago
    130ms is considered a lot in competitives FPS. But for videoconference, I don't know if it's a lot, In the fps you get feedback of your own actions instantly, in videconference, you don't get this feedback. When you talk to someone in person, there is a latency, the time the brain process what happened in front of him. It add a latency of at least 250/350ms. I don't think adding 130ms on top of that is pathetic.
- jakear 6y ago
  Pretty bold to dismiss someone’s work as “pathetic”, especially if you can’t demonstrate you’ve done better.
- dang 6y ago
  Please don't slam other people's work like that on HN. We're trying for curious conversation here, and if you're that dismissive, it's a bummer and spoils conversation.
  - snvzz 6y ago
    Dismissive how? If it wasn't clear, I am happy about such an achievement in the reduction of latency. But 130ms, while much better than 500ms, is still bad. And I pointed outside the project as likely culprits. It's a stretch to claim my post "slams other people's work".
    - dang 6y ago
      You've edited it since I replied—in fact you've largely rewritten it. It's a bit dodgy to do that and then ask "Dismissive how?" as if you hadn't just changed what people were reacting to. Generally speaking, once there are replies, the courteous thing to do is to leave the original context intact and add edits in a way that makes clear what the edits are. Otherwise you pull the carpet out from under the replies, which will no longer make sense in the way they did when they were posted. I was reacting to the word "pathetic" in your original version. In the absence of disambiguating information about your intent [1], it sounded mean. If you had posted originally what you've got there now, it wouldn't have been a problem. [1] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20burden%20disambiguate&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
      - snvzz 6y ago
        >you've edited it since I replied—in fact you've largely rewritten it. Yes, I edited it to make more positive, based on the reaction. It apparently was seen as as attacking the project, when that was far from my intent. >I was reacting to your use of the word "pathetic". I noticed this, and changed as soon as I could. I do believe 130ms really is pathetic and shows there's much work to be done (probably outside of the project, which I thought was clear in my original post but as per its reception, it definitely wasn't, so I expanded that part to put more weight on that), but do absolutely appreciate and welcome pretty much any work on reducing latency (endemic throughout software and hardware; everybody thinks 1ms here and there isn't important and it all adds up).
mbrubeck 6y ago
Previously: https://news.ycombinator.com/item?id=23540586 https://news.ycombinator.com/item?id=23540586
deleted 6y ago
[deleted]

38 points by lukastyrychtr 6y ago

130ms is a world better than 500ms and a much welcome improvement, but it is still terrible. Latency happens throughout the whole stack; Unfortunately much would need to be fixed outside this project to achieve any further significant improvement. Operating System, firmware, blackbox hardware are some other non-negligible sources of latency. Everything adds up.

sgillen 6y ago
Is it? They claim to be state of the art in this space. They are processing(?) and sending 3k video over the network, I think 130ms sounds pretty good.
- snvzz 6y ago
  130ms means 8 whole frames behind at 60fps. You'll have to add the latency of the network on top of that. My point isn't to dismiss their work (any work on reducing latency is very welcome), but rather to highlight that 130ms is still bad, and that the rest of the stack (OS, firmware, blackbox hardware) is definitely a problem.
dan-robertson 6y ago
The post claims they achieved that by controlling the whole stack
- snvzz 6y ago
  I saw that, but I still saw their "capture drivers" configuration example. As I understand it, they have little in terms of outside dependencies, but still do run their software on a common OS. Unlike seL4[0], Linux/OSX/Windows are unsuitable for real time. [0]: https://sel4.systems/About/seL4-whitepaper.pdf https://sel4.systems/About/seL4-whitepaper.pdf
Carpetsmoker 6y ago
What would a "non-pathetic" latency look like? What improvements in "the stack" would be needed? The 130ms is "the time from light hitting the camera to when it appears on-screen on the other side". The article doesn't really specify where the latency happens, but with just the encoding/decoding of video alone I think you'll very quickly get latencies in roughly the ~100ms ballpark no matter what.
- ponker 6y ago
  Not “no matter what,” but perhaps “running on a multitasking operating system.” With a hardware encoder and 1-frame buffer you could get down to 50ms.
  - vertex-four 6y ago
    Across the internet, at all times?
    - ponker 6y ago
      No, just exclusive of transport. So 50ms in-the-building and another 50ms for SF-NY and another 50ms for SF->Moscow or similar.
  - snvzz 6y ago
    As anyone that ever used AmigaOS would tell you, multitasking is not an excuse for poor latency behavior. seL4[0] will guarantee (with formal proof) a low maximum latency despite multitasking. [0]: https://sel4.systems/About/seL4-whitepaper.pdf https://sel4.systems/About/seL4-whitepaper.pdf
- Kuinox 6y ago
  130ms is considered a lot in competitives FPS. But for videoconference, I don't know if it's a lot, In the fps you get feedback of your own actions instantly, in videconference, you don't get this feedback. When you talk to someone in person, there is a latency, the time the brain process what happened in front of him. It add a latency of at least 250/350ms. I don't think adding 130ms on top of that is pathetic.
jakear 6y ago
Pretty bold to dismiss someone’s work as “pathetic”, especially if you can’t demonstrate you’ve done better.
dang 6y ago
Please don't slam other people's work like that on HN. We're trying for curious conversation here, and if you're that dismissive, it's a bummer and spoils conversation.
- snvzz 6y ago
  Dismissive how? If it wasn't clear, I am happy about such an achievement in the reduction of latency. But 130ms, while much better than 500ms, is still bad. And I pointed outside the project as likely culprits. It's a stretch to claim my post "slams other people's work".
  - dang 6y ago
    You've edited it since I replied—in fact you've largely rewritten it. It's a bit dodgy to do that and then ask "Dismissive how?" as if you hadn't just changed what people were reacting to. Generally speaking, once there are replies, the courteous thing to do is to leave the original context intact and add edits in a way that makes clear what the edits are. Otherwise you pull the carpet out from under the replies, which will no longer make sense in the way they did when they were posted. I was reacting to the word "pathetic" in your original version. In the absence of disambiguating information about your intent [1], it sounded mean. If you had posted originally what you've got there now, it wouldn't have been a problem. [1] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20burden%20disambiguate&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
    - snvzz 6y ago
      >you've edited it since I replied—in fact you've largely rewritten it. Yes, I edited it to make more positive, based on the reaction. It apparently was seen as as attacking the project, when that was far from my intent. >I was reacting to your use of the word "pathetic". I noticed this, and changed as soon as I could. I do believe 130ms really is pathetic and shows there's much work to be done (probably outside of the project, which I thought was clear in my original post but as per its reception, it definitely wasn't, so I expanded that part to put more weight on that), but do absolutely appreciate and welcome pretty much any work on reducing latency (endemic throughout software and hardware; everybody thinks 1ms here and there isn't important and it all adds up).

mbrubeck 6y ago

Previously: https://news.ycombinator.com/item?id=23540586 https://news.ycombinator.com/item?id=23540586

deleted 6y ago

[deleted]