4 ms·

Casual book scanning could benefit from stereo cameras, if there's a way for iPhone apps to access this data. As a mathematician I've long wanted to figure out

Casual book scanning could benefit from stereo cameras, if there's a way for iPhone apps to access this data. As a mathematician I've long wanted to figure out the code to match up several photographs for a 3D model of an open book, to correctly flatten out the image of each page. Bonus points to use AI to clean up the image, or convert to source that faithfully represents the page layout. There are apps that take one photo and then guess how to correct the distortion. They don't do as good a job as one could do with the actual 3D data. Many of us don't want to dedicate the space or money for a professional book scanner, and can't find an affordable flatbed scanner that scans all the way to one edge to better support books. I'll sometimes slice a second used copy of a cookbook, to scan every page using a sheetfed scanner. Grabbing one's phone should win ease-of-use over any other solution, if it could match the quality of these other approaches. 3D cameras would help that.

hnuser123456 3y ago
Given that the distortion is always only going to be in the X dimension, unless the page is crumpled, could you just try to detect straight horizontal lines on the page, come up with the distortion to straighten them, and apply that to the whole page?
- Syzygies 3y ago
  I don't agree with your "X dimension" assumption. I've seen the poor performance of current phone apps. My point is that one could do better for less coding effort with access to a 3D model. Either a pair of photos, or a 3D camera.
  - iamflimflam1 3y ago
    Agreed, most basic approaches just assume that a page is a rectangle and that you just need to apply perspective correction. Something that could map the 3D surface onto a sheet would be awesome.
- KolenCh 3y ago
  I’d guess, without knowing the details, this is already how it is done? Guessing can only take you so far, not to mention some pages has no straight lines (and I think you need orthogonal lines for the reconstruction to be effective?) and some has deceivingly straight lines (or even slanted lines.) What is described above is something I’ve seen in a documentary. They have a very fancy camera and the person holding the book can just slowly flipping through the pages, and it can reconstruct the flat pages. I guess the problem is not technology or mathematical model behind it, but how to make it cheap. It is analogous to Google’s self driving car, that you want to solve the same problem with much cheaper technology.
ajanuary 3y ago
Have you tried the document scanner feature in the notes app?
turquoisevar 3y ago
I’m not sure if that data will be available on iPhone. Still, I understand that on the Vision Pro, none of this data (and similar data) will be available to developers due to privacy concerns. That said, you might be able to do this already with iDevices equipped with LiDAR (e.g., iPhones Pro and iPads Pro). They’re already used to create 3D objects, but I suppose no one has used them yet to take the next step you are talking about.

hnuser123456 3y ago

Given that the distortion is always only going to be in the X dimension, unless the page is crumpled, could you just try to detect straight horizontal lines on the page, come up with the distortion to straighten them, and apply that to the whole page?

Syzygies 3y ago
I don't agree with your "X dimension" assumption. I've seen the poor performance of current phone apps. My point is that one could do better for less coding effort with access to a 3D model. Either a pair of photos, or a 3D camera.
- iamflimflam1 3y ago
  Agreed, most basic approaches just assume that a page is a rectangle and that you just need to apply perspective correction. Something that could map the 3D surface onto a sheet would be awesome.
KolenCh 3y ago
I’d guess, without knowing the details, this is already how it is done? Guessing can only take you so far, not to mention some pages has no straight lines (and I think you need orthogonal lines for the reconstruction to be effective?) and some has deceivingly straight lines (or even slanted lines.) What is described above is something I’ve seen in a documentary. They have a very fancy camera and the person holding the book can just slowly flipping through the pages, and it can reconstruct the flat pages. I guess the problem is not technology or mathematical model behind it, but how to make it cheap. It is analogous to Google’s self driving car, that you want to solve the same problem with much cheaper technology.

ajanuary 3y ago

Have you tried the document scanner feature in the notes app?

turquoisevar 3y ago

I’m not sure if that data will be available on iPhone. Still, I understand that on the Vision Pro, none of this data (and similar data) will be available to developers due to privacy concerns. That said, you might be able to do this already with iDevices equipped with LiDAR (e.g., iPhones Pro and iPads Pro). They’re already used to create 3D objects, but I suppose no one has used them yet to take the next step you are talking about.