6 ms·
Citation? I've read over 50 papers in MISR and SISR (I wrote a lit review) and I have never seen mention of actual hardware that would shift the imaging system;
by anon_tor_12345 6y ago
Citation? I've read over 50 papers in MISR and SISR (I wrote a lit review) and I have never seen mention of actual hardware that would shift the imaging system; indeed such a system would have been my dissertation topic if I hadn't switched areas.
- CarVac 6y agoCertain Hasselblad cameras have had this for a long time, and Pentax, Olympus, Panasonic, and Sony have it on various models, using the sensor shift image stabilization to implement it.
- galago 6y agoI think what he's talking about is what Sony calls Pixel Shift Multi Shooting. Other camera manufacturers do it too. https://support.d-imaging.sony.co.jp/support/ilc/psms/ilce7rm3/en/index.html https://support.d-imaging.sony.co.jp/support/ilc/psms/ilce7r...
- anon_tor_12345 6y agoWow that is in fact exactly it. I'm actually quite impressed they're able to shift the sensor array by exactly one pixel (or even close to) since that's on the order of microns.
- brudgers 6y agoPentax did it with the K70 back in 2016.
- yfkar 6y agoThey also did it earlier in 2015 with Pentax K-3 II. Also Olympus OM-D E-M5 Mark II had a similar feature in 2015.
- Stratoscope 6y agoIn fact the Olympus implementation of this feature moves the sensor by half a photosite diagonally. They do it by re-purposing the existing in-body stabilization mechanism to move the sensor around. User 'twic' posted a link to a very interesting article that describes this and also explains the difference between photosites and pixels: https://chriseyrewalker.com/the-hi-res-mode-of-the-olympus-om-d-e-m1-mark-ii/ https://chriseyrewalker.com/the-hi-res-mode-of-the-olympus-o...
- RobLach 6y agoSome projector manufacturers are shifting a 4K DMD around 4 times to increase the refresh rate of a 1080P image 4x.
- azornathogron 6y agoPiezo actuators can make very fine movements. You can get positioning systems that can adjust position by nanometers, even sub-nm [1]. Also, cameras have had stabilization systems for a while now; I would assume they need similar pixel-scale precision. Some cameras shift the lens, some shift the sensor, but either way they need to shift the image-on-sensor by a very small amount, and also do it very rapidly. [1] For example: https://www.pro-lite.co.uk/File/psj_piezoelectric_nanopositioning.php https://www.pro-lite.co.uk/File/psj_piezoelectric_nanopositi...
- deleted 6y ago[deleted]
- bobtail3 6y agoProbably not exactly what he is talking about, but this also sounds similar to dithering. Where with repeated measurements and random noise you can statistically estimate the value of a signal below the quantization level.
- Thrymr 6y agoIndeed, it includes things like the Drizzle algorithm that has been used by Hubble space telescope astronomers for a while: https://www.stsci.edu/ftp/science/hdf/combination/drizzle.html https://www.stsci.edu/ftp/science/hdf/combination/drizzle.ht...
- wongarsu 6y agoSynthetic aperture radar [1][2] uses this principle. In that case the imaging system is fixed on a moving aircraft or satellite. I think in general satellite imaging is a good place to look for such implementations, since they have a naturally and predictably moving imaging system. 1: https://en.wikipedia.org/wiki/Synthetic-aperture_radar https://en.wikipedia.org/wiki/Synthetic-aperture_radar 2: https://www.youtube.com/watch?v=u2bUKEi9It4 https://www.youtube.com/watch?v=u2bUKEi9It4
- anon_tor_12345 6y agoSAR is similar but not the same since there you're super-resolving in time rather than space. Also in that instance it's just conventional MISR since you're not driving the imaging system (more information is being passively collected as the targets passes).
- ska 6y agoSAR is similar to the video technique mentioned, agree it's not quite the same but if underlying assumptions hold still more estimate than "guess".
- oivey 6y agoI think it’s more useful to think in terms of how well sampled the observations are relative to the size of the output space. SISR is very undersampled, and MISR is oversampled. SAR reconstruction techniques can fall in either bucket.
- mNovak 6y agoA major difference here is that SAR uses phase information, whereas to my knowledge optical techniques are not doing that.
- twic 6y agoI just bought a camera that does it, so it definitely exists. https://chriseyrewalker.com/the-hi-res-mode-of-the-olympus-om-d-e-m1-mark-ii/ https://chriseyrewalker.com/the-hi-res-mode-of-the-olympus-o...
- Stratoscope 6y agoThat's a really informative article; thanks for posting it. I also have an Olympus E-M1 MkII, but I haven't tried the high resolution mode yet. You just gave me a TODO item!
- twic 6y agoI haven't tried it yet either. It really needs a tripod, and i don't have one.
- shock-value 6y agoMost modern mirrorless cameras have such hardware (usually denoted "in-body image stabilization"). The "repurposing" of it to acquire multiple captures at various slight offsets and combine them intelligently is maybe a bit more recent, but still pretty common at this point. https://en.wikipedia.org/wiki/Image_stabilization#Sensor-shift https://en.wikipedia.org/wiki/Image_stabilization#Sensor-shi... https://support.d-imaging.sony.co.jp/support/ilc/psms/ilce7rm3/en/index.html https://support.d-imaging.sony.co.jp/support/ilc/psms/ilce7r... https://www.nikonimgsupport.com/na/NSG_article?articleNo=000042799&configured=1&lang=en_SG https://www.nikonimgsupport.com/na/NSG_article?articleNo=000... https://www.canon.co.uk/pro/stories/8-stops-image-stabilization/ https://www.canon.co.uk/pro/stories/8-stops-image-stabilizat...
- avereveard 6y agoI think the technique was used a lot prior in astronomy, where I encountered it the first time. There is called stacking, and the subpixel offset comes naturally from path distortion in the atmosphere itself. I remember using registax when the first batch of consumer telezoom point and click camera came out and got a 30x long exposure of the moon, with, well, mediocre results.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- dev_tty01 6y agoThe iPhone 12 Pro physically shifts the sensor for image stabilization, so precise positioning is possible.
- vlovich123 6y agoYou can also do it in software by just recording IMU with high precision time stamps (a lot of camera sensors have a gpio they can interrupt on even every scan line) and post-processing. There’s cool techniques where they can remove various issues in rolling shutter through this technique to get global-shutter like quality and removing camera-movement induced motion blur. I haven’t heard of it applied to super res but I don’t see why not. I think Google uses similar techniques to implement their software HDR solution which takes 3 back to back snapshots at different exposure levels and merges them.
- bayindirh 6y agoNewer Sony mirrorless cameras embed gyroscope data into videos for further stabilization of the image when IBIS is not enough Theoretically, with 30FPS cameras like Sony A1 and said gyroscope data, you can create super resolution images. IIRC Olympus' some handheld super resolution modes use both shake and sensor shift to increase resolution.
- sorenjan 6y agoGoogle uses it in Google camera. They're not shifting the sensor themselves, but they take advantage of the camera shake users introduce by taking handheld photos. https://ai.googleblog.com/2018/10/see-better-and-further-with-super-res.html https://ai.googleblog.com/2018/10/see-better-and-further-wit...
- anon_tor_12345 6y agoYes I'm very familiar with this but this is just MISR i.e. purely a software solution (Peyman Milanfar is one of the original researchers associated with MISR). Fortunately elsewhere in this thread hardware implementations have been demonstrated.
- sorenjan 6y agoThere are hardware implementations referenced in the blog post, and in the linked published paper. > In the early 2000s, Farsiu et al. [2006] and Gotoh and Okutomi [2004]formulated superresolution from arbitrary motion as an optimization problem that would be infeasible for interactive rates. Ben-Ezraet al. [2005] created a jitter camera prototype to do super-resolution using controlled subpixel detector shifts. This and other works inspired some commercial cameras (e.g.,Sony A6000,Pentax FF K1,Olympus OM-D E-M1orPanasonic Lumix DC-G9) to adopt multi-frame techniques, using controlled pixel shifting of the physical sensor. However, these approaches require the use of a tripod or a static scene. https://arxiv.org/pdf/1905.03277.pdf https://arxiv.org/pdf/1905.03277.pdf
- anon_tor_12345 6y agodamn the Ben-Ezra paper is exactly what i imagined being the scope of work for my planned dissertation. guess it's a good thing i didn't pursue it lol.
- gnopgnip 6y agoPentax DLSR have this feature too. The same motors that are used to control the sensor and prevent camera shake are used to offset the sensor slightly while multiple images are taken