6 ms·
Wow, this is a great idea! Looking at the video, looks like the latency isn't great; what kind of problems should be solved for this to have a ~100ms latency?
by pcr910303 7y ago
Wow, this is a great idea!
Looking at the video, looks like the latency isn't great; what kind of problems should be solved for this to have a ~100ms latency?
Rewriting it in C/C++/Rust instead of Python would be one... Any ideas?
- drchewbacca 7y agoIt looks like it's just applying a bunch of filters/convolutions to the image. You could write it in C++ to be pretty quick, in CUDA you could get it down to microseconds I think. Though that does not take account of the latency of the webcam itself.
- fortran77 7y agoToo bad you can't get CUDA easily on a recent Macintosh.
- tantalor 7y agoHuh? https://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html https://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-...
- RussianCow 7y agoMacBooks haven't had Nvidia GPUs for a few years.
- coldtea 7y agoThey have also supported eGPUS for years though...
- codetrotter 7y agoYou’ll also need an NVIDIA GPU. All of the MacBooks that came with NVIDIA GPUs are from 2010 and earlier. That’s quite a while ago in computer age. https://support.apple.com/en-us/HT204349#nvidia https://support.apple.com/en-us/HT204349#nvidia
- nacs 7y agoYou could use one via the eGPU support in most newer Macs: https://support.apple.com/en-us/HT208544 https://support.apple.com/en-us/HT208544
- codetrotter 7y agoYeah but then you haven’t turned a MacBook into a Touchscreen with $1 of Hardware. You’ve turned a MacBook into a Touchscreen with $price_of_GPU + $price_of_eGPU_adapter + $1. And additionally you’ve made your MacBook a lot less portable.
- fortran77 7y agoI'm always amused when Mac people ask me why I don't use Macs and I tell them "I program CUDA for a living" and they respond "But you can jerry rig a PCIx slot in a box over a wonky cable plug it into your Thunderfire port" or something. As if that's a real solution.
- applecrazy 7y agoThere’s no need to do any of that. Commercially-available eGPU boxes[1] work well, no jank required. [1]: https://www.sonnettech.com/product/egfx-breakaway-box.html https://www.sonnettech.com/product/egfx-breakaway-box.html
- codetrotter 7y agoI know we frown upon the kind of comment I’m about to make but I really feel the need to point this out right here; relevant username!
- liability 7y agoCommercially available jank is still jank. And the matter of portability (why else would you be using a laptop in the first place?) still remains. Forget walking around with it around towb in a bag; if I am trying to work on my deck chair outside, where do I put that mother of all dongles? Balance it precariously on the chair's armrest? No thanks!
- fortran77 7y agoHuh? You have to go back two versions of the OS in order to follow those instructions.
- seandougall 7y agoYou can do all that in Metal -- though as GP said, it doesn't matter, because there's fixed latency at the camera input.
- snazz 7y agoWebcams usually have somewhat poor latency, at least in my experience. And I think the performance critical code is all in OpenCV, not Python.
- megous 7y agoWebcams don't have memory, so they can't have latency. It's all in the software. In fact, I've done some apps on Linux that just did readout of the frame from the camera sensor to ram, and displayed the buffer on the next framebuffer flip with zero copies. There was no perceived latency. It just takes some work.
- monocasa 7y agoWeb cams do have memory, and do a remarkable amount of post processing on the frames, hence the latency. Source: have used used webcam cube cams in a robotics application where latency needed to be accounted for.
- throwaway5752 7y agoNot to drag this too far off topic, but what kind of post-processing? And is it standardized across different types of webcams?
- jrowley 7y agoWhile not post-processing, often you can configure the exposure duration and frame rate which will make things feel more or less responsive.
- simias 7y agoNoise reduction is a big one. You usually need to interpolate over several frames to do that. Also depending on the bus being used cameras often compress the image using JPEG to reduce the bandwith (very common with USB cameras, I doubt that the Mac camera does that). Then on a computer you have the latency of whatever pipeline is being used and how much buffering is involved. For instance since typically the display doesn't run exactly synchronized with the camera you'll have some frames that'll end up "stuck" waiting for the monitor's vsync. If the app then does some internal buffering on top of that you can very easily reach a noticeable amount of latency. In general you can consider than anything above 100ms is easily noticeable and that's only 3 frames at 30fps and 6 at 60, it's really not that much when you consider the complexity of modern video capture pipelines.
- Liquix 7y agoYMMV, but I've experienced slight webcam lag on Macbooks even with Apple-optimized applications such as Photo Booth and FaceTime. Perhaps the biggest bottleneck is at the hardware / driver level instead of at the CPU-cycles level?
- JustSomeNobody 7y ago> Rewriting it in C/C++/Rust instead of Python would be one... Would it? It's using libraries for the heavy lifting already.
- collyw 7y agoI had a laptop with a touchscreen and the only time I ever used it was when the linux mouse drivers were playing up. So I guess that would be one use.
- ghostbrainalpha 7y agoThey are asking what kind of problems need to be solved in order to speed up the latency. They are not asking what kind of problems having a touch screen on a laptop would solve.
- sdegutis 7y agoSlightly off topic, but C/C++/Rust are always the go-to languages when talking about high performance. People love the speed of C but it's so bare-bones that they move to C++, or it's too dangerous so they move to Rust. But those are much too complicated, so people came up with Zig and Nim and D instead, but those are very far from C, even though they're not as far as C++ and Rust. So I'm wondering, are there any languages almost as close to C but slightly safer and slightly less bare-bones? Because I like C a lot, I think it's a great language with an excellent simplicity, except for some slight rough edges.
- coldtea 7y agoUnfortunately no. We could have a better/safer C but alas we don't. It's either C or all the way to Zig/Rust/etc territory (or using C++ like C).
- nicoburns 7y agoZig seems very close to C to me. What makes you think it's different?
- blattimwind 7y agoThe gap between C's model and a somewhat memory safe language is unfortunately very large. You can either limit semantics a lot, use GC (mostly-D, Go, ...) or have relatively complex ways to describe life times (Rust).
- eclat 7y agoCrystal? Hasn't reached 1.0 though.
- snazz 7y agoI think Crystal is garbage-collected, right?
- rafaelvasco 7y agoYeah, the whole language is designed around it. Impossible to remove without rewriting it from scratch: https://github.com/crystal-lang/crystal/wiki/FAQ#language-x-has-feature-y-why-dont-you-have-such-feature https://github.com/crystal-lang/crystal/wiki/FAQ#language-x-...
- TYPE_FASTER 7y agoI'd be interested in a version that used the Vision Framework. Back when I looked at the Apple frameworks for iOS, they utilized the Neural Engine hardware available on iPhones while OpenCV didn't (that may have changed). The laptop hardware is different of course, but it would be an interesting performance comparison. Here's an interesting overview of using Core frameworks with Python: https://developer.apple.com/videos/play/wwdc2018/719/ https://developer.apple.com/videos/play/wwdc2018/719/