3 ms·
The idea itself is great, but the quality of the result obviously depends highly on the type of hardware - namely the spectra of the display and camera. An LCD
by nom 10y ago
The idea itself is great, but the quality of the result obviously depends highly on the type of hardware - namely the spectra of the display and camera.
An LCD has a totally different spectrum (broad) than an LED display (single wavelength). Cameras on the other hand always have a broad spectrum, but it varies depending on the color filters, the response curve of the pixels as well as the post processing (e.g. the demosaicing algorithm). Regarding response curve of the display: that's also something they have to calibrate for and it's not only a hardware parameter because it can be modified in software, for example by the iOS Night Shift feature.
Making this work with a single piece of hardware is hard enough and I wonder if they'll target the android market at all. My guess is they're going to release it only for iPhones because it's so homogeneous.
Also, the press release photo made me giggle, the front camera isn't even facing the object they are scanning :D
Edit: Saying LEDs have a single wavelength is probably not correct, but they have a very narrow band depending on the technology used.
- smallnamespace 10y agoCouldn't you in theory calibrate this for any feasible device using a known light source and a large enough set of color swatches with known response curves? Inferring device parameters should just be solving some system of equations given enough constraints, and if you overconstrain the system you also get an estimate of how much noise is in your parameters. In this case, you need to simultaneously estimate the LED wavelengths as well as the response curve of the camera. Not saying it'd necessarily be worthwhile to do, but if that actually worked, all you'd need to do is send a light source + a color swatch booklet through the mail to a potential user if the device isn't in the database yet.
- dave_sullivan 10y ago> using a known light source and a large enough set of color swatches with known response curves? Inferring device parameters should just be solving some system of equations given enough constraints, and if you overconstrain the system you also get an estimate of how much noise is in your parameters. If that's true, and those constraints aren't terribly well understood, it would make for a good machine learning problem. Fwiw.
- smallnamespace 10y agoSorry I wasn't being clear -- the RGB camera response to each patch of a swatch should give (up to) 3 different constraints under normal lighting, plus another 3 by cycling through LED colors, so in theory you only need ceil((k+3) / 6) colors to estimate a k-variable camera response curve (the other 3 estimate your LED wavelengths).
- nom 10y agoGood idea. I think they _must_ provide some sort of calibration to make it work with unknown devices. I can imagine that you can construct a special color swatch with known spectral responses, but it totally depends on how the algorithm itself works. The only thing we can say for sure is that the quality of the result varies from device to device - if it works at all.