3 ms·
The app doesn't have to do this itself, phones do it for you. ARCore on Android, ARKit on iOS.
by dmd 2mo ago
The app doesn't have to do this itself, phones do it for you. ARCore on Android, ARKit on iOS.
- hahahaa 2mo agoOh yeah I mean how do the phones do that. I guess it uses dead reckoning with visual cues and some machine learning.
- dmd 2mo ago"SLAM" is the term to google.
- deleted 2mo ago[deleted]
- boothby 2mo agoYour phone has an accelerometer, which can be integrated twice to result in slightly accurate dead reckoning; just keep accumulating 3-vectors with every time-slice. That's a helluva lot easier than visual cues.
- addaon 2mo ago"slightly accurate" is the correct description, though -- dead reckoning on a cell-phone-class IMU accumulates drift too quickly to keep objects anchored in space for more than seconds at a time. You need something (visual odometry, SLAM) that provides long-term stability, even if it has short-term noise or poor availability, to fuse in.
- gregsadetsky 2mo agoWhen this topic comes up, I'm always happy to share this incredible lecture from many many years ago about sensor fusion and dead reckoning and how you _cannot_ integrate acceleration twice: https://www.youtube.com/watch?v=C7JQ7Rpwn2k&t=1401s https://www.youtube.com/watch?v=C7JQ7Rpwn2k&t=1401s (the whole video is incredible, but this timestamp is the magical bit)
- pezezin 2mo ago12 years ago I had to write the firmware for an IMU. It took me some real effort to convince my bosses that double integration was, for all practical purposes, impossible. I wish I had known about this video back then... IMU are very useful to known your orientation though. Even an inaccurate IMU is useful to provide an initial estimate to feed vision-based algorithms.
- hahahaa 2mo agoSo you tend to need additional data or assumptions to constrain the double integral?
- pezezin 2mo agoYou don't double integrate, period. You essentially have three options: 1) An absolute positioning system like GPS. 2) A visual positioning system like fiducial markers or SLAM. 3) Something that can measure linear speed and then you integrate the speed only once. Easy to do on vehicles, but even the most accurate system will quickly drift.
- forsalebypwner 2mo agoon iPhone 12 Pro Max and later (maybe only on Pros, I forget) they have a LiDAR sensor built in, allowing for impressively accurate 3D scans of objects with a free app like Scaniverse. But yeah I think AR was the intention there
- aplavin 2mo agoIn my experience, on Android, it definitely relies on camera: AR doesn't work in dark room, and barely works in slightly-lit room. So it must be fusing camera + accelerometer + ...