4 ms·

So they're using the screen consisting of three monochromatic LED types as a full-spectrum light source? I don't think that works... You still just get three p

by notnot 10y ago

So they're using the screen consisting of three monochromatic LED types as a full-spectrum light source? I don't think that works... You still just get three points on the spectrum, same as the Bayer filter on the camera. I suppose if the three screen LED wavelengths were significantly different from the three camera filter wavelengths then you could: Illuminate with screen Red to get Rscreen. Illuminate with screen Green to get Gscreen. Illuminate with screen Blue to get Bscreen. Use ambient full-spectrum light to get Rfilter, Gfilter, and Bfilter. Then you'd have 6 points on the spectrum.

danbruc 10y ago
That was also what I was thinking. But given that neither the LEDs nor the Bayer filters have an actual line spectrum it might be possible to obtain some more information than under the assumption that they are ideal line spectra. Could there also be some non-linearities? Does the spectrum of a LED change somewhat depending on the input power?
- nom 10y ago
  I think LEDs have a much more narrow band spectrum. At least the colored ones.
nom 10y ago
A Bayer filter doesn't give you single wavelengths they have a broad, somewhat overlapping spectrum. An LED display on the other hand has three distinct narrow bands. I'm not sure how their inverse algorithm could work, but I have a feeling it should be possible to get more than three points of the spectrum by displaying multiple light patterns. Regarding full-spectrum ambient light: they can't use it at all because they have to subtract it from the images. You can only recover spectral information from the light you control. At least that's what I'm thinking right now.
highd 10y ago
I think you're correct, barring significant nonlinearity in the bayer mask or object. Technically you can get 9 linearly independent points - each combination of light channels on with each combination of bayer mask channels. Ideally only 3 of those will be nonzero, but if the bayer mask is imperfect you'll see some illumination on adjacent channels. Environmental background is subtracted out from all since it's unknown, so that doesn't give another point. There's also no way you're measuring pesticide residue with that - I doubt that would even be possible with a high-end visible hyperspectral camera. Maybe with a raman spectrometer. I've designed a couple versions of cell phone camera-based spectrometers and spectral imagers, so I'm relatively familiar with the design principles.
- nom 10y ago
  Can you give us some insight on the whole pesticide thing? How is it possible to detect it with a spectrometer in general? Are you sure that it is impossible to detect it, even if the app is 'calibrated' to a certain object, e.g. an apple? I think if you limit the search space it could be possible!?
  - highd 10y ago
    The pesticide probably doesn't look very different in the visible spectrum at that low of a concentration - otherwise its presence would be pretty easily detectable to the human eye. If you require detection capabilities exceeding the human eye you're going to need a much more sensitive setup. Most chemicals have characteristic spectra in the infrared, so you'd need sensors going to much larger wavelengths to have significant difference - even then it would be hard to detect against the variation in signal from fruit.
  - kortex 10y ago
    It's impossible. Most chemicals of interest are pretty boring in the visible spectrum. I'd say >95% of pure substances I've worked with - everything from pesticides to pharmaceuticals - are some variant on "white to off-white solid" or "clear to amber liquid." White/clear indicates that all photons visible to us interact with the materially equally. You get tans, yellows, and browns largely from high-frequency (deep blue/purple part of the spectrum) being absorbed by assorted chemical bonds. Spectroscopy is predominantly done with UV (200-280 nm most common) and IR, which are regions where photonic interaction is dominated by electronic and vibration/rotational transitions, respectively. Visible light absorption is typically caused by highly conjugated bonds and metal-coordination complexes. In terms of day-to-day, this is almost exclusively dyes (synthetic and natural). Dyes also tend to be really potent absorbers - you only need minuscule amounts of them to create very vivid colors. So a purely visible-light-based app would at best be able to give you a handle of what sort of dyes are in something. It won't tell you if it has pesticides (let alone traces!) or HFCS or nutrients or what-have-you. tl;dr - no, it's not remotely possible to even detect pesticides with visible light.
mozumder 10y ago
This is far from hyperspectral imaging, where you're supposed to have 256 or 1024 bands of colors per pixel.

by notnot 10y ago

danbruc 10y ago

That was also what I was thinking. But given that neither the LEDs nor the Bayer filters have an actual line spectrum it might be possible to obtain some more information than under the assumption that they are ideal line spectra. Could there also be some non-linearities? Does the spectrum of a LED change somewhat depending on the input power?

nom 10y ago
I think LEDs have a much more narrow band spectrum. At least the colored ones.

nom 10y ago

A Bayer filter doesn't give you single wavelengths they have a broad, somewhat overlapping spectrum. An LED display on the other hand has three distinct narrow bands. I'm not sure how their inverse algorithm could work, but I have a feeling it should be possible to get more than three points of the spectrum by displaying multiple light patterns. Regarding full-spectrum ambient light: they can't use it at all because they have to subtract it from the images. You can only recover spectral information from the light you control. At least that's what I'm thinking right now.

highd 10y ago

I think you're correct, barring significant nonlinearity in the bayer mask or object. Technically you can get 9 linearly independent points - each combination of light channels on with each combination of bayer mask channels. Ideally only 3 of those will be nonzero, but if the bayer mask is imperfect you'll see some illumination on adjacent channels. Environmental background is subtracted out from all since it's unknown, so that doesn't give another point. There's also no way you're measuring pesticide residue with that - I doubt that would even be possible with a high-end visible hyperspectral camera. Maybe with a raman spectrometer. I've designed a couple versions of cell phone camera-based spectrometers and spectral imagers, so I'm relatively familiar with the design principles.

nom 10y ago
Can you give us some insight on the whole pesticide thing? How is it possible to detect it with a spectrometer in general? Are you sure that it is impossible to detect it, even if the app is 'calibrated' to a certain object, e.g. an apple? I think if you limit the search space it could be possible!?
- highd 10y ago
  The pesticide probably doesn't look very different in the visible spectrum at that low of a concentration - otherwise its presence would be pretty easily detectable to the human eye. If you require detection capabilities exceeding the human eye you're going to need a much more sensitive setup. Most chemicals have characteristic spectra in the infrared, so you'd need sensors going to much larger wavelengths to have significant difference - even then it would be hard to detect against the variation in signal from fruit.
- kortex 10y ago
  It's impossible. Most chemicals of interest are pretty boring in the visible spectrum. I'd say >95% of pure substances I've worked with - everything from pesticides to pharmaceuticals - are some variant on "white to off-white solid" or "clear to amber liquid." White/clear indicates that all photons visible to us interact with the materially equally. You get tans, yellows, and browns largely from high-frequency (deep blue/purple part of the spectrum) being absorbed by assorted chemical bonds. Spectroscopy is predominantly done with UV (200-280 nm most common) and IR, which are regions where photonic interaction is dominated by electronic and vibration/rotational transitions, respectively. Visible light absorption is typically caused by highly conjugated bonds and metal-coordination complexes. In terms of day-to-day, this is almost exclusively dyes (synthetic and natural). Dyes also tend to be really potent absorbers - you only need minuscule amounts of them to create very vivid colors. So a purely visible-light-based app would at best be able to give you a handle of what sort of dyes are in something. It won't tell you if it has pesticides (let alone traces!) or HFCS or nutrients or what-have-you. tl;dr - no, it's not remotely possible to even detect pesticides with visible light.

mozumder 10y ago

This is far from hyperspectral imaging, where you're supposed to have 256 or 1024 bands of colors per pixel.