3 ms·

Never understood why my GCardboard couldn’t do that, my phone sure has a bunch of accelerometers and giros. Sure higher and other techs can track better but isn

Never understood why my GCardboard couldn’t do that, my phone sure has a bunch of accelerometers and giros. Sure higher and other techs can track better but isn’t it enough for a basic sense of mouvement? For most of the applications I won’t more than a few meter anyway. Probably some have tried and I’ll be curious to know what prevent it.

jsheard 2y ago
The problem with accelerometers and gyros is they drift badly if you try to derive absolute positioning from them alone. They need to be fused with some other form of tracking to anchor them in absolute space, which in the case of the Quest and Vision Pro is done with multiple outward-facing cameras fed into a SLAM algorithm. Maybe Cardboard could have attempted to use the phones camera for SLAM, but a single lens would only have got them so far. Dedicated VR headsets have at least four cameras pointing in different directions, which are sometimes augmented by IR projectors and/or LiDAR.
- bee_rider 2y ago
  Most phones have a couple cameras nowadays… I think the Pro iPhones (some, at least) even have some sort of lidar system that seems like it ought to be helpful? Anyway, it is a shame, I guess the market must not have been there.
  - Joel_Mckay 2y ago
    Most phones use a rolling shutter, so doing machine vision for low-latency motion/pose is difficult or unfeasible on a mobile cpu. Best regards =3
    - ben_w 2y ago
      Surely use the accelerometers for real-time/low latency, only use the cameras to correct for drift? Not that I've tried for VR, but I did already create AR demos 5-6 years ago.
      - Joel_Mckay 2y ago
        In general VR is easier for sure, as in AR the latency issues manifest quicker. The sensor fusion of IMUs is usually not stable very long, but it does mitigate a few of the very noticeable problems with camera trackers. Still, it always boils down to the power budget... having a 2.6 kg head mounted unit is ludicrous. lol =)
- Joel_Mckay 2y ago
  LADAR/3D-cameras or LIDAR are both expensive parts with limited capabilities. Note rapid pose-recovery using cameras and or SLAM has been tried, but again people end up pooching the CPU/power budget.. and rolling camera shutters are useless... difficult to deploy as a wearable tech. A few years back, we did design a set of <160USD parts to get repeatable absolute head and controller spacial location/pose to sub +-3mm in a room. The key was being able to resolve stable _absolute_ pose at >24Hz with <10kiB/s of low-latency data to handle. i.e. a small generic mcu _quickly_ handles the dual kalman filters and IMU sensors fusion, and battery life is reasonable. Now build your own versions, it is not that hard... ask Alphabet/Meta/Apple... lol... Those new 3D lenticular screens look pretty cool, but the prices are still not for consumer hardware yet. Best of luck =3
- bigiain 2y ago
  Lots of quadcopter flight controllers use 9DOF IMUs , with 3 gyros, 3 accelerometers, and 3 compasses. The absolute directional data from the compasses solves (at least most of) the angular/gyro drift. The translational drift is harder for VR/AR headsets indoors. Drones can do sensor fusion with GPS and the accelerometers to solve translational drift from the accelerometers (or, for FPV drones, they just let the meatware compensate).
  - RF_Savage 2y ago
    The "9DoF" in IMU datasheets is a marketing term, they just add up all the sensor dimensions they have. Some IMU modules talk about "10DoF" , because they have added a barometer to it. So even a good "9DoF" IMU is not usable for 6DoF VR, as it still drifts way too much. Sadly the magnetometer in the IMUs suffers from all the magnetic fields generated by the rest of the electronics around it. This might also be one of the reasons why 9DoF IMUs are increasingly rare on the market.
    - deleted 2y ago
      [deleted]
- needle0 2y ago
  To be pedantic, two cameras were enough for the headset to track itself (eg. Lenovo Mirage Solo). The reason that headsets nowadays have 4 cameras is for it to also track the hand controllers that are being held by the user and being flung around nearby...which this also seems to lack.
foobarbecue 2y ago
Dead reckoning using MEMS IMUs accumulates error way too fast.
- adgjlsfhk1 2y ago
  even if you supplement with GPS?
  - jsheard 2y ago
    That works if you're building a cruise missile, but not so much if you need millimeter accuracy indoors.
    - aziaziazi 2y ago
      Ah! So it could 6DoF if I run outdoor fast enough with MarathonSimulator
nox101 2y ago
especially given the camera, it seems like you could do some kind of motion tracking. I guess a Quest has 4 cameras for motion tracking so 1 isn't enough. Though maybe putting a 180degree wide angle lens over it would let it do the work for 4?
grumbel 2y ago
See "Pure IMU-based Positional Tracking is a No-go"[1], a position from an IMU starts to drift in a fraction of a second and than shoots off into infinity. Without an absolute reference there is nothing you can do to stop it and the errors will accumulate. It's not just a little bad, it's completely unusable. A further complication is that the IMUs you find in phones, especially back when all this was new, weren't very good. So even a plain 3DoF experience would suffer from slow response and yaw-drift from lack of magnetometer, which is why GearVR had an extra IMU in the headset and why Daydream required new certified phones with better IMUs. 3DoF can still be good enough for VR180/360 movies, which don't allow movement to begin with. But as far as gaming is concerned, you really do need 6DoF, as even a bit of leaning forward in a chair will make it instantly obvious that the headset motion and the headset visuals are out of sync. [1] https://www.youtube.com/watch?v=_q_8d0E3tDk https://www.youtube.com/watch?v=_q_8d0E3tDk

jsheard 2y ago

The problem with accelerometers and gyros is they drift badly if you try to derive absolute positioning from them alone. They need to be fused with some other form of tracking to anchor them in absolute space, which in the case of the Quest and Vision Pro is done with multiple outward-facing cameras fed into a SLAM algorithm. Maybe Cardboard could have attempted to use the phones camera for SLAM, but a single lens would only have got them so far. Dedicated VR headsets have at least four cameras pointing in different directions, which are sometimes augmented by IR projectors and/or LiDAR.

bee_rider 2y ago
Most phones have a couple cameras nowadays… I think the Pro iPhones (some, at least) even have some sort of lidar system that seems like it ought to be helpful? Anyway, it is a shame, I guess the market must not have been there.
- Joel_Mckay 2y ago
  Most phones use a rolling shutter, so doing machine vision for low-latency motion/pose is difficult or unfeasible on a mobile cpu. Best regards =3
  - ben_w 2y ago
    Surely use the accelerometers for real-time/low latency, only use the cameras to correct for drift? Not that I've tried for VR, but I did already create AR demos 5-6 years ago.
    - Joel_Mckay 2y ago
      In general VR is easier for sure, as in AR the latency issues manifest quicker. The sensor fusion of IMUs is usually not stable very long, but it does mitigate a few of the very noticeable problems with camera trackers. Still, it always boils down to the power budget... having a 2.6 kg head mounted unit is ludicrous. lol =)
Joel_Mckay 2y ago
LADAR/3D-cameras or LIDAR are both expensive parts with limited capabilities. Note rapid pose-recovery using cameras and or SLAM has been tried, but again people end up pooching the CPU/power budget.. and rolling camera shutters are useless... difficult to deploy as a wearable tech. A few years back, we did design a set of <160USD parts to get repeatable absolute head and controller spacial location/pose to sub +-3mm in a room. The key was being able to resolve stable _absolute_ pose at >24Hz with <10kiB/s of low-latency data to handle. i.e. a small generic mcu _quickly_ handles the dual kalman filters and IMU sensors fusion, and battery life is reasonable. Now build your own versions, it is not that hard... ask Alphabet/Meta/Apple... lol... Those new 3D lenticular screens look pretty cool, but the prices are still not for consumer hardware yet. Best of luck =3
bigiain 2y ago
Lots of quadcopter flight controllers use 9DOF IMUs , with 3 gyros, 3 accelerometers, and 3 compasses. The absolute directional data from the compasses solves (at least most of) the angular/gyro drift. The translational drift is harder for VR/AR headsets indoors. Drones can do sensor fusion with GPS and the accelerometers to solve translational drift from the accelerometers (or, for FPV drones, they just let the meatware compensate).
- RF_Savage 2y ago
  The "9DoF" in IMU datasheets is a marketing term, they just add up all the sensor dimensions they have. Some IMU modules talk about "10DoF" , because they have added a barometer to it. So even a good "9DoF" IMU is not usable for 6DoF VR, as it still drifts way too much. Sadly the magnetometer in the IMUs suffers from all the magnetic fields generated by the rest of the electronics around it. This might also be one of the reasons why 9DoF IMUs are increasingly rare on the market.
  - deleted 2y ago
    [deleted]
needle0 2y ago
To be pedantic, two cameras were enough for the headset to track itself (eg. Lenovo Mirage Solo). The reason that headsets nowadays have 4 cameras is for it to also track the hand controllers that are being held by the user and being flung around nearby...which this also seems to lack.

foobarbecue 2y ago

Dead reckoning using MEMS IMUs accumulates error way too fast.

adgjlsfhk1 2y ago
even if you supplement with GPS?
- jsheard 2y ago
  That works if you're building a cruise missile, but not so much if you need millimeter accuracy indoors.
  - aziaziazi 2y ago
    Ah! So it could 6DoF if I run outdoor fast enough with MarathonSimulator

nox101 2y ago

especially given the camera, it seems like you could do some kind of motion tracking. I guess a Quest has 4 cameras for motion tracking so 1 isn't enough. Though maybe putting a 180degree wide angle lens over it would let it do the work for 4?

grumbel 2y ago

See "Pure IMU-based Positional Tracking is a No-go"[1], a position from an IMU starts to drift in a fraction of a second and than shoots off into infinity. Without an absolute reference there is nothing you can do to stop it and the errors will accumulate. It's not just a little bad, it's completely unusable. A further complication is that the IMUs you find in phones, especially back when all this was new, weren't very good. So even a plain 3DoF experience would suffer from slow response and yaw-drift from lack of magnetometer, which is why GearVR had an extra IMU in the headset and why Daydream required new certified phones with better IMUs. 3DoF can still be good enough for VR180/360 movies, which don't allow movement to begin with. But as far as gaming is concerned, you really do need 6DoF, as even a bit of leaning forward in a chair will make it instantly obvious that the headset motion and the headset visuals are out of sync. [1] https://www.youtube.com/watch?v=_q_8d0E3tDk https://www.youtube.com/watch?v=_q_8d0E3tDk