3 ms·

The problem with accelerometers and gyros is they drift badly if you try to derive absolute positioning from them alone. They need to be fused with some other f

The problem with accelerometers and gyros is they drift badly if you try to derive absolute positioning from them alone. They need to be fused with some other form of tracking to anchor them in absolute space, which in the case of the Quest and Vision Pro is done with multiple outward-facing cameras fed into a SLAM algorithm. Maybe Cardboard could have attempted to use the phones camera for SLAM, but a single lens would only have got them so far. Dedicated VR headsets have at least four cameras pointing in different directions, which are sometimes augmented by IR projectors and/or LiDAR.

bee_rider 2y ago
Most phones have a couple cameras nowadays… I think the Pro iPhones (some, at least) even have some sort of lidar system that seems like it ought to be helpful? Anyway, it is a shame, I guess the market must not have been there.
- Joel_Mckay 2y ago
  Most phones use a rolling shutter, so doing machine vision for low-latency motion/pose is difficult or unfeasible on a mobile cpu. Best regards =3
  - ben_w 2y ago
    Surely use the accelerometers for real-time/low latency, only use the cameras to correct for drift? Not that I've tried for VR, but I did already create AR demos 5-6 years ago.
    - Joel_Mckay 2y ago
      In general VR is easier for sure, as in AR the latency issues manifest quicker. The sensor fusion of IMUs is usually not stable very long, but it does mitigate a few of the very noticeable problems with camera trackers. Still, it always boils down to the power budget... having a 2.6 kg head mounted unit is ludicrous. lol =)
Joel_Mckay 2y ago
LADAR/3D-cameras or LIDAR are both expensive parts with limited capabilities. Note rapid pose-recovery using cameras and or SLAM has been tried, but again people end up pooching the CPU/power budget.. and rolling camera shutters are useless... difficult to deploy as a wearable tech. A few years back, we did design a set of <160USD parts to get repeatable absolute head and controller spacial location/pose to sub +-3mm in a room. The key was being able to resolve stable _absolute_ pose at >24Hz with <10kiB/s of low-latency data to handle. i.e. a small generic mcu _quickly_ handles the dual kalman filters and IMU sensors fusion, and battery life is reasonable. Now build your own versions, it is not that hard... ask Alphabet/Meta/Apple... lol... Those new 3D lenticular screens look pretty cool, but the prices are still not for consumer hardware yet. Best of luck =3
bigiain 2y ago
Lots of quadcopter flight controllers use 9DOF IMUs , with 3 gyros, 3 accelerometers, and 3 compasses. The absolute directional data from the compasses solves (at least most of) the angular/gyro drift. The translational drift is harder for VR/AR headsets indoors. Drones can do sensor fusion with GPS and the accelerometers to solve translational drift from the accelerometers (or, for FPV drones, they just let the meatware compensate).
- RF_Savage 2y ago
  The "9DoF" in IMU datasheets is a marketing term, they just add up all the sensor dimensions they have. Some IMU modules talk about "10DoF" , because they have added a barometer to it. So even a good "9DoF" IMU is not usable for 6DoF VR, as it still drifts way too much. Sadly the magnetometer in the IMUs suffers from all the magnetic fields generated by the rest of the electronics around it. This might also be one of the reasons why 9DoF IMUs are increasingly rare on the market.
  - deleted 2y ago
    [deleted]
needle0 2y ago
To be pedantic, two cameras were enough for the headset to track itself (eg. Lenovo Mirage Solo). The reason that headsets nowadays have 4 cameras is for it to also track the hand controllers that are being held by the user and being flung around nearby...which this also seems to lack.

bee_rider 2y ago

Most phones have a couple cameras nowadays… I think the Pro iPhones (some, at least) even have some sort of lidar system that seems like it ought to be helpful? Anyway, it is a shame, I guess the market must not have been there.

Joel_Mckay 2y ago
Most phones use a rolling shutter, so doing machine vision for low-latency motion/pose is difficult or unfeasible on a mobile cpu. Best regards =3
- ben_w 2y ago
  Surely use the accelerometers for real-time/low latency, only use the cameras to correct for drift? Not that I've tried for VR, but I did already create AR demos 5-6 years ago.
  - Joel_Mckay 2y ago
    In general VR is easier for sure, as in AR the latency issues manifest quicker. The sensor fusion of IMUs is usually not stable very long, but it does mitigate a few of the very noticeable problems with camera trackers. Still, it always boils down to the power budget... having a 2.6 kg head mounted unit is ludicrous. lol =)

Joel_Mckay 2y ago

LADAR/3D-cameras or LIDAR are both expensive parts with limited capabilities. Note rapid pose-recovery using cameras and or SLAM has been tried, but again people end up pooching the CPU/power budget.. and rolling camera shutters are useless... difficult to deploy as a wearable tech. A few years back, we did design a set of <160USD parts to get repeatable absolute head and controller spacial location/pose to sub +-3mm in a room. The key was being able to resolve stable _absolute_ pose at >24Hz with <10kiB/s of low-latency data to handle. i.e. a small generic mcu _quickly_ handles the dual kalman filters and IMU sensors fusion, and battery life is reasonable. Now build your own versions, it is not that hard... ask Alphabet/Meta/Apple... lol... Those new 3D lenticular screens look pretty cool, but the prices are still not for consumer hardware yet. Best of luck =3

bigiain 2y ago

Lots of quadcopter flight controllers use 9DOF IMUs , with 3 gyros, 3 accelerometers, and 3 compasses. The absolute directional data from the compasses solves (at least most of) the angular/gyro drift. The translational drift is harder for VR/AR headsets indoors. Drones can do sensor fusion with GPS and the accelerometers to solve translational drift from the accelerometers (or, for FPV drones, they just let the meatware compensate).

RF_Savage 2y ago
The "9DoF" in IMU datasheets is a marketing term, they just add up all the sensor dimensions they have. Some IMU modules talk about "10DoF" , because they have added a barometer to it. So even a good "9DoF" IMU is not usable for 6DoF VR, as it still drifts way too much. Sadly the magnetometer in the IMUs suffers from all the magnetic fields generated by the rest of the electronics around it. This might also be one of the reasons why 9DoF IMUs are increasingly rare on the market.
- deleted 2y ago
  [deleted]

needle0 2y ago

To be pedantic, two cameras were enough for the headset to track itself (eg. Lenovo Mirage Solo). The reason that headsets nowadays have 4 cameras is for it to also track the hand controllers that are being held by the user and being flung around nearby...which this also seems to lack.