5 ms·

Here is a video of it in action: https://www.youtube.com/watch?v=owSJK7vMSLk https://www.youtube.com/watch?v=owSJK7vMSLk Interestingly, humans come very close

Here is a video of it in action: https://www.youtube.com/watch?v=owSJK7vMSLk https://www.youtube.com/watch?v=owSJK7vMSLk Interestingly, humans come very close to it during operation.

dugditches 2y ago
Not much different than everyday on sites everywhere with boom cranes. Which are arguably much more dangerous due to being less dexterous and no video. Notably 1:23 where it's similar to working with a crane.
roughly 2y ago
That looks incredibly cool, and also instantly nausea inducing - I can’t imagine the video feed and the head gyros are fast enough to avoid that little bit of lag and judder that turn VR into a vomit machine. The feedback on the arm is very cool, though.
- gpm 2y ago
  FPV drone hardware has commercialized very cheap very low latency video feeds to headsets. Not with headtracking though. Not sure that's reasonably possible.
  - yial 2y ago
    FPV headsets can have head tracking. I don’t know cost now, but it was possible even in 2012/2013/2014 when I worked in that industry.
    - gpm 2y ago
      Huh, so there is, with physical servos moving a camera. Any idea what latency numbers might look like? Seems unlikely the physical movement can be especially low latency.
    - unrealhoang 2y ago
      What's the advantages of head tracking though? I don't see it's in anyway better than just joystick to control robot head movement.
      - yial 2y ago
        You’re already using two joysticks with your thumbs or thumbs and index to fly.
      - SoftTalker 2y ago
        It's a completely natural movement for the operator, that will require almost no practice or training, and also leaves the operator's hands free to operate other controls.
- namibj 2y ago
  Why not? There's no need for the neck to have more than 1~5 ms of lag, and about 5~10 ms photon lag even with a framebuffer involved (if you align the rolling shutter of the camera to the row-by-row multiplexed display, say OLED @ 240Hz, you should manage 4~5 ms).
- PoignardAzur 2y ago
  I wonder if it wouldn't be easier / more comfortable for the worker to just get a low-poly reconstructed 3D rendering of the environment. That way at least they could move their head as much as they want and still get a read-time feed in their goggle without the usual VR tricks.
  - pixl97 2y ago
    Because said low res rendering will remove very important cues that workers repairing stuff need to see, like potential things cracks in materials.
- fragmede 2y ago
  just over so it on the cameras so there isn't a latency because you're not actually moving the giant robot head
- enva2712 2y ago
  Responses so far say to optimize neck latency or use a joystick but why not use a 360 camera and handle it in reprojection. First pass could literally be a gopro or insta360 streaming to the headset and you could eventually integrate cameras and do the stitching yourself. The tech already exists and works, and it removes moving parts and actuators
- theshackleford 2y ago
  > that little bit of lag and judder that turn VR into a vomit machine. This doesn’t apply to everyone, so it’s just a matter of training people with more tolerance.
omoikane 2y ago
I like how the robot head motion is linked to the operator's head motion, such that the robot is able to nod its head near 2:15. I also like the feedback control near 0:35.
downboots 2y ago
now start collecting user movement data with the visual recordings and we can eventually automate all the things

Here is a video of it in action: https://www.youtube.com/watch?v=owSJK7vMSLk https://www.youtube.com/watch?v=owSJK7vMSLk Interestingly, humans come very close to it during operation.

dugditches 2y ago

Not much different than everyday on sites everywhere with boom cranes. Which are arguably much more dangerous due to being less dexterous and no video. Notably 1:23 where it's similar to working with a crane.

roughly 2y ago

That looks incredibly cool, and also instantly nausea inducing - I can’t imagine the video feed and the head gyros are fast enough to avoid that little bit of lag and judder that turn VR into a vomit machine. The feedback on the arm is very cool, though.

gpm 2y ago
FPV drone hardware has commercialized very cheap very low latency video feeds to headsets. Not with headtracking though. Not sure that's reasonably possible.
- yial 2y ago
  FPV headsets can have head tracking. I don’t know cost now, but it was possible even in 2012/2013/2014 when I worked in that industry.
  - gpm 2y ago
    Huh, so there is, with physical servos moving a camera. Any idea what latency numbers might look like? Seems unlikely the physical movement can be especially low latency.
  - unrealhoang 2y ago
    What's the advantages of head tracking though? I don't see it's in anyway better than just joystick to control robot head movement.
    - yial 2y ago
      You’re already using two joysticks with your thumbs or thumbs and index to fly.
    - SoftTalker 2y ago
      It's a completely natural movement for the operator, that will require almost no practice or training, and also leaves the operator's hands free to operate other controls.
namibj 2y ago
Why not? There's no need for the neck to have more than 1~5 ms of lag, and about 5~10 ms photon lag even with a framebuffer involved (if you align the rolling shutter of the camera to the row-by-row multiplexed display, say OLED @ 240Hz, you should manage 4~5 ms).
PoignardAzur 2y ago
I wonder if it wouldn't be easier / more comfortable for the worker to just get a low-poly reconstructed 3D rendering of the environment. That way at least they could move their head as much as they want and still get a read-time feed in their goggle without the usual VR tricks.
- pixl97 2y ago
  Because said low res rendering will remove very important cues that workers repairing stuff need to see, like potential things cracks in materials.
fragmede 2y ago
just over so it on the cameras so there isn't a latency because you're not actually moving the giant robot head
enva2712 2y ago
Responses so far say to optimize neck latency or use a joystick but why not use a 360 camera and handle it in reprojection. First pass could literally be a gopro or insta360 streaming to the headset and you could eventually integrate cameras and do the stitching yourself. The tech already exists and works, and it removes moving parts and actuators
theshackleford 2y ago
> that little bit of lag and judder that turn VR into a vomit machine. This doesn’t apply to everyone, so it’s just a matter of training people with more tolerance.

omoikane 2y ago

I like how the robot head motion is linked to the operator's head motion, such that the robot is able to nod its head near 2:15. I also like the feedback control near 0:35.

downboots 2y ago

now start collecting user movement data with the visual recordings and we can eventually automate all the things