4 ms·
Well, first some clarification - "edge" means "on robot" versus something in the cloud. And the reason you do this is latency and connectivity. I am designing
by TaylorAlexander 7y ago
Well, first some clarification - "edge" means "on robot" versus something in the cloud. And the reason you do this is latency and connectivity.
I am designing a four wheel drive robot using the NVIDIA AGX Xavier [1] that will follow trails on its own or follow the operator on trails. You don't want your robot to lose cellular coverage and become useless. Even if you had coverage, there would be significant data usage as Rover uses four 4k cameras, which is about 30 megapixels (actually they max out at 13mp each or 52mp total). Constantly streaming that to the cloud would be very expensive on a metered internet connection. Even on a direct line the machine would saturate many broadband connections. Of course you can selectively stream but this makes things more complicated.
Latency is an issue. Imagine a self driving car that required a cloud connection. It's approaching an intersection and someone on a bicycle falls over near its path. Better send that sensor data to the cloud fast to determine how to act!
On my Rover robot it streams the cameras directly in to the GPU memory where it can be processed using ML without ever being copied through the CPU. It's super low latency and allows for robots that respond rapidly to their environment. Imagine trying to make a ping-pong playing robot with a cloud connection.
I am also designing a farming robot. [2] We don't expect any internet connection on farms!
[1] https://reboot.love/t/new-cameras-on-rover/ https://reboot.love/t/new-cameras-on-rover/
[2] https://www.twistedfields.com/technology https://www.twistedfields.com/technology
Edit: Don’t forget security! Streaming high resolution sensors over the cloud is a security nightmare.
- fizixer 7y agoEdge means on-premise (on robot) as you said. But 'edge,' as used in context of AI, is also a wink-and-a-nod that the device is inference-only (no learning, no training). The term "inference only" doesn't sound very marketing-friendly.
- my123 7y agoAGX Xavier can do training on device just fine - and run every CUDA workload. It's just not the fastest device at that, you'd prefer a desktop GPU if you can for such a purpose.
- michaelt 7y agoI assume what fizixer means is, if you're making an Amazon-Alexa-type-thing, training 1 model on 1 million user's data will work better than 1 million models trained on 1 user's data each. AFAIK the "Roomba learns the layout of your house" type of edge learning is generally done with SLAM rather than neural networks. There might be other applications for edge learning, of course.
- epmaybe 7y agoThis is a bit off topic, but I'm constantly looking at ways to efficiently stream 4K cameras live to local displays as well as remote displays at the highest framerate and resolution possible. How feasible would it be on the xavier to stream 2 4k cameras and display them on at least 2 4k screens? Extra points if you could do that and simultaneously upload to a streaming service, such as twitch.
- m463 7y agoI've wondered about this too. I think the magic camera interconnect is CSI/CSI2 and it's not really flexible enough. You either have really short copper interconnects, or unavailable fiber interconnects. What would be cool is if csi to ethernet were a thing. either low latency put-it-on-the-wire or compressed. I don't know, maybe it is. But make it a standard like rca jacks.
- achuwilson 7y agoIt is possible to extend CSI through good HDMI cables https://www.robotshop.com/en/arducam-csi-hdmi-cable-extension-module-2pk.html https://www.robotshop.com/en/arducam-csi-hdmi-cable-extensio...
- manofmanysmiles 7y agoYou can buy kits that send video over coax including power for somewhat reasonable prices: https://leopardimaging.com/product/nvidia-jetson-cameras/nvidia_agx_xavier_fpdlinkiii_camera_kits/li-ar0231-ti953-xavier/ https://leopardimaging.com/product/nvidia-jetson-cameras/nvi... I haven't tried them, but am considered them for project.
- TaylorAlexander 7y agoYou can certainly do this using machine vision cameras. Either USB3, Gig Ethernet, or CSI interface (16cm max run length I believe). I forget how best to attach two displays to the Xavier but that’s seems doable. I got my cameras from e-consystems and they’ve got some USB3 cameras that could do it. At least I’m pretty sure. My USB3 cameras just showed up and I haven’t tried them yet.
- jgalt212 7y agoYes, but the GPU is needed for training so "we" do this on prem or in the cloud. But the actual prediction computations are fast. Do we really need an edge GPU for these?
- TaylorAlexander 7y agoYes. An edge GPU is required to run inference on high resolution sensor data for many robotics applications. Running inference is faster than training but it’s not all “fast”. Inference on CPU can take many seconds with large image operations. Not useful for certain robotics applications.