4 ms·

Most Android phones don't have a good USB controller though, as it's not considered a priority. Without it you'll be restricted to the blocky MJPEG garbage with

Most Android phones don't have a good USB controller though, as it's not considered a priority. Without it you'll be restricted to the blocky MJPEG garbage with terribly downsampled chroma squeezed through the USB 2.0 connection, and get the quality equivalent to a laptop webcam.

ClassyJacket 3y ago
480mbps isn't enough for good quality video? It's 3.75x the maximum bitrate of a 4K Blu Ray. It should be fine. It can't be lossless, but it should be damn high quality.
- orbital-decay 3y ago
  I was going to say something about MJPEG, but it looks like UVC supports H.264 for a decade already. I guess I was confused by the cheap USB 2.0 webcams, they mostly use MJPEG and yes it's utter trash. Still, good webcams use USB 3.0 and no chroma subsampling, to avoid recompression and enable certain tricks. I hope the YUV 4:4:4 mode will be supported in this, for those who have the speed and want chromakeys/non-blurry reds.
- duskwuff 3y ago
  > 480mbps isn't enough for good quality video? It's 3.75x the maximum bitrate of a 4K Blu Ray. Webcams need to be very near real-time to be useful. That severely limits what kinds of compression you can use on the video stream.
  - ClassyJacket 3y ago
    That's absolutely true but I still very much doubt it's so bad that you would exceed 480mbps even on a decent resolution stream. We're not talking multiple orders of magnitude difference. Like... if all else fails just send every frame as a separate image with a fast encoder and you've still got 1.6 megabytes for each. Steam in-home streaming needs to be realtime too, but is very playable, and most people aren't realistically getting 480mbps out of their wifi. 4k I don't know... 1080p no problem. Don't most phones have dedicated hardware video encoders on their SoC now anyway?
    - rcme 3y ago
      You could transmit raw rgba pixels at 1920x1080 and 60 fps in less than 500 mbps.
      - ClassyJacket 3y ago
        I don't think so... my maths (for RGB, no need for alpha channel) comes out to 2,847mbps. Did you accidentally switch megabits and megabytes?
        duskwuff 3y ago
        And don't forget that 480 Mbps is the line rate. Practical throughput to a device will be substantially less, especially if other devices are on the bus.
- GuB-42 3y ago
  To make a point, early webcams were USB 1.0, 12Mbps maximum, and it was enough for SD-quality video. USB 2.0 is 40x faster, which, assuming similar encoding (i.e. 20 year old tech), should be more than enough for 1080p video. It should be enough to best any laptop webcam, which are usually USB2 internally, but with worse optics and without the processing power of a smartphone behind it. Using more fancy encoding, 4k should be no problem, even with a bit of headroom for on-device encoding and low latency.
  - Dalewyn 3y ago
    >without the processing power of a smartphone behind it. Considering the cheapest of desktop/laptop CPUs and iGPUs are more powerful than the best mobile CPUs and GPUs, this is a problem with the encoders and video chat software involved.
    - Retric 3y ago
      The best cellphone CPU’s easily beat cheap laptops’s from a few years ago that used low end 10th gen Intel chips. They don’t have the terminal budget to quite beat today’s chips, but it’s much closer than you might expect. That might seem unfair, but many people replace their phones far more frequently than their computers.
    - GuB-42 3y ago
      I meant the chip between the builtin laptop webcam and its internal USB2 connection, which is typically a tiny, dedicated IC. Smartphones cameras are not USB2, they are, I think, directly connected to a powerful SoC. The USB2 link is for sending data to the computer after the smartphone has processed it with all the might of its main processor.
    - kccqzy 3y ago
      Have you been living under a rock? The encoders have been hardware based for more than a decade.
- ianburrell 3y ago
  The video out isn't over USB2 data but over the high speed lines that are for USB3 and alternate mode. It should be possible to have DisplayPort output but with USB2 data. In fact, we know it is possible because the iPhone 15 does it. It has USB2 data speeds but 4K Displayport.
- kccqzy 3y ago
  Indeed Apple CarPlay is entirely based on sending H.264 video over IP over USB 2.0 and it's very good quality with low lag. It's absolutely good enough for webcam style usage. https://devstreaming-cdn.apple.com/videos/wwdc/2016/722x2eefo3u2rp8k8qs/722/722_developing_carplay_systems_part_1.pdf?dl=1 https://devstreaming-cdn.apple.com/videos/wwdc/2016/722x2eef...
gsich 3y ago
This is not as bad as you make it sound. The cheapest USB 2.0 capture card uses MJPEG 4:2:0 and it's excellent for webcam usage. The newer 3.0 device has 4:2:2 YUV.
- orbital-decay 3y ago
  Depends on what you call webcam usage I guess, people have different needs. If you want any color grading or simply have deep shadows in the frame, MJPEG will make it look really bad. If you need chroma keying, it's extremely sensitive to compression artifacts and chroma resolution. MJPEG blockiness is going to be amplified (and if you have noise then blocks will be flickering), you'll be forced to wear a hat because it will cut your hair otherwise, and the edge of anything red (skintones, your shirt) will be smudged or terribly pixelated against the green screen. More expensive webcams like the Facecam Pro have the uncompressed or really high bitrate mode for this reason.
  - gsich 3y ago
    This might be implementation dependent. MJPEG per se is not bad as you make it out to be. Example reviews: https://www.youtube.com/watch?v=daS5RHVAl2U https://www.youtube.com/watch?v=daS5RHVAl2U and many more. They all boil down to the same thing: excellent device, even with MJPEG. Obviously it's more dependent on what you use as camera, than the encoding method.
    - orbital-decay 3y ago
      I'm talking about webcams, which have more complex use cases than unprocessed game capture. MJPEG is necessary blocky and bad at low values (shadows); if you're going to chat with friends, streaming it directly, it's probably not going to be noticeable. But if you want to look good, any raw noise or any processing will amplify the compression artifacts, especially in the dark tones, and 4:2:0 will wreck the saturated reds against the opposite background. Take a look at this OBS forums thread I found, which is a pretty good illustration. [1] This is the best key from a cheap MJPEG webcam I've ever seen, usually it's terrible and streamers have to wear hats. The edge quality in the final picture is still bad (especially hair), and that's with unholy amount of polishing the turd he did, sacrificing latency and probably holding his breath (figuratively) to not go outside of his finely tuned setup. Especially the temporal noise reduction. Check this frame out as well [2], and pay attention to his shirt and hand - the edge is terribly pixelated, which he points out as well. And that is probably the best you can do with the crappy compressed source. This is the reason cameras like Facecam Pro exist at all. Uncompressed YUV with full chroma resolution makes it all far easier. > Obviously it's more dependent on what you use as camera, than the encoding method. You can get away with a cheap camera, if you're using it indoors with controlled lighting; you can't get away with bad compression if you want to process it. [1] https://obsproject.com/forum/threads/151600/ https://obsproject.com/forum/threads/151600/ [2] https://obsproject.com/forum/attachments/78631/ https://obsproject.com/forum/attachments/78631/
bigger_inside 3y ago
he existing webcam apps for android, like droidcam, send the signal through the local wifi when the app is installed on the phone and the PC.
bmicraft 3y ago
If you can record 1 minute of video on your phone and the resulting file has a size of less than 2GB, then you're not even close to saturating USB 2.0's effective rate of ~42MB/s (=2.5GB/min) while streaming to a pc. My phone needs 483MB/min with those settings.

ClassyJacket 3y ago

480mbps isn't enough for good quality video? It's 3.75x the maximum bitrate of a 4K Blu Ray. It should be fine. It can't be lossless, but it should be damn high quality.

orbital-decay 3y ago
I was going to say something about MJPEG, but it looks like UVC supports H.264 for a decade already. I guess I was confused by the cheap USB 2.0 webcams, they mostly use MJPEG and yes it's utter trash. Still, good webcams use USB 3.0 and no chroma subsampling, to avoid recompression and enable certain tricks. I hope the YUV 4:4:4 mode will be supported in this, for those who have the speed and want chromakeys/non-blurry reds.
duskwuff 3y ago
> 480mbps isn't enough for good quality video? It's 3.75x the maximum bitrate of a 4K Blu Ray. Webcams need to be very near real-time to be useful. That severely limits what kinds of compression you can use on the video stream.
- ClassyJacket 3y ago
  That's absolutely true but I still very much doubt it's so bad that you would exceed 480mbps even on a decent resolution stream. We're not talking multiple orders of magnitude difference. Like... if all else fails just send every frame as a separate image with a fast encoder and you've still got 1.6 megabytes for each. Steam in-home streaming needs to be realtime too, but is very playable, and most people aren't realistically getting 480mbps out of their wifi. 4k I don't know... 1080p no problem. Don't most phones have dedicated hardware video encoders on their SoC now anyway?
  - rcme 3y ago
    You could transmit raw rgba pixels at 1920x1080 and 60 fps in less than 500 mbps.
    - ClassyJacket 3y ago
      I don't think so... my maths (for RGB, no need for alpha channel) comes out to 2,847mbps. Did you accidentally switch megabits and megabytes?
      - duskwuff 3y ago
        And don't forget that 480 Mbps is the line rate. Practical throughput to a device will be substantially less, especially if other devices are on the bus.
GuB-42 3y ago
To make a point, early webcams were USB 1.0, 12Mbps maximum, and it was enough for SD-quality video. USB 2.0 is 40x faster, which, assuming similar encoding (i.e. 20 year old tech), should be more than enough for 1080p video. It should be enough to best any laptop webcam, which are usually USB2 internally, but with worse optics and without the processing power of a smartphone behind it. Using more fancy encoding, 4k should be no problem, even with a bit of headroom for on-device encoding and low latency.
- Dalewyn 3y ago
  >without the processing power of a smartphone behind it. Considering the cheapest of desktop/laptop CPUs and iGPUs are more powerful than the best mobile CPUs and GPUs, this is a problem with the encoders and video chat software involved.
  - Retric 3y ago
    The best cellphone CPU’s easily beat cheap laptops’s from a few years ago that used low end 10th gen Intel chips. They don’t have the terminal budget to quite beat today’s chips, but it’s much closer than you might expect. That might seem unfair, but many people replace their phones far more frequently than their computers.
  - GuB-42 3y ago
    I meant the chip between the builtin laptop webcam and its internal USB2 connection, which is typically a tiny, dedicated IC. Smartphones cameras are not USB2, they are, I think, directly connected to a powerful SoC. The USB2 link is for sending data to the computer after the smartphone has processed it with all the might of its main processor.
  - kccqzy 3y ago
    Have you been living under a rock? The encoders have been hardware based for more than a decade.
ianburrell 3y ago
The video out isn't over USB2 data but over the high speed lines that are for USB3 and alternate mode. It should be possible to have DisplayPort output but with USB2 data. In fact, we know it is possible because the iPhone 15 does it. It has USB2 data speeds but 4K Displayport.
kccqzy 3y ago
Indeed Apple CarPlay is entirely based on sending H.264 video over IP over USB 2.0 and it's very good quality with low lag. It's absolutely good enough for webcam style usage. https://devstreaming-cdn.apple.com/videos/wwdc/2016/722x2eefo3u2rp8k8qs/722/722_developing_carplay_systems_part_1.pdf?dl=1 https://devstreaming-cdn.apple.com/videos/wwdc/2016/722x2eef...

gsich 3y ago

This is not as bad as you make it sound. The cheapest USB 2.0 capture card uses MJPEG 4:2:0 and it's excellent for webcam usage. The newer 3.0 device has 4:2:2 YUV.

orbital-decay 3y ago
Depends on what you call webcam usage I guess, people have different needs. If you want any color grading or simply have deep shadows in the frame, MJPEG will make it look really bad. If you need chroma keying, it's extremely sensitive to compression artifacts and chroma resolution. MJPEG blockiness is going to be amplified (and if you have noise then blocks will be flickering), you'll be forced to wear a hat because it will cut your hair otherwise, and the edge of anything red (skintones, your shirt) will be smudged or terribly pixelated against the green screen. More expensive webcams like the Facecam Pro have the uncompressed or really high bitrate mode for this reason.
- gsich 3y ago
  This might be implementation dependent. MJPEG per se is not bad as you make it out to be. Example reviews: https://www.youtube.com/watch?v=daS5RHVAl2U https://www.youtube.com/watch?v=daS5RHVAl2U and many more. They all boil down to the same thing: excellent device, even with MJPEG. Obviously it's more dependent on what you use as camera, than the encoding method.
  - orbital-decay 3y ago
    I'm talking about webcams, which have more complex use cases than unprocessed game capture. MJPEG is necessary blocky and bad at low values (shadows); if you're going to chat with friends, streaming it directly, it's probably not going to be noticeable. But if you want to look good, any raw noise or any processing will amplify the compression artifacts, especially in the dark tones, and 4:2:0 will wreck the saturated reds against the opposite background. Take a look at this OBS forums thread I found, which is a pretty good illustration. [1] This is the best key from a cheap MJPEG webcam I've ever seen, usually it's terrible and streamers have to wear hats. The edge quality in the final picture is still bad (especially hair), and that's with unholy amount of polishing the turd he did, sacrificing latency and probably holding his breath (figuratively) to not go outside of his finely tuned setup. Especially the temporal noise reduction. Check this frame out as well [2], and pay attention to his shirt and hand - the edge is terribly pixelated, which he points out as well. And that is probably the best you can do with the crappy compressed source. This is the reason cameras like Facecam Pro exist at all. Uncompressed YUV with full chroma resolution makes it all far easier. > Obviously it's more dependent on what you use as camera, than the encoding method. You can get away with a cheap camera, if you're using it indoors with controlled lighting; you can't get away with bad compression if you want to process it. [1] https://obsproject.com/forum/threads/151600/ https://obsproject.com/forum/threads/151600/ [2] https://obsproject.com/forum/attachments/78631/ https://obsproject.com/forum/attachments/78631/

bigger_inside 3y ago

he existing webcam apps for android, like droidcam, send the signal through the local wifi when the app is installed on the phone and the PC.

bmicraft 3y ago

If you can record 1 minute of video on your phone and the resulting file has a size of less than 2GB, then you're not even close to saturating USB 2.0's effective rate of ~42MB/s (=2.5GB/min) while streaming to a pc. My phone needs 483MB/min with those settings.