12 ms·
Night Sight for Pixel phones
- londons_explore 8y agoPrior work at Google Research before it made it into the product: https://ai.googleblog.com/2017/04/experimental-nighttime-photography-with.html https://ai.googleblog.com/2017/04/experimental-nighttime-pho... And by the original researcher in 2016: https://www.youtube.com/watch?v=S7lbnMd56Ys https://www.youtube.com/watch?v=S7lbnMd56Ys
- yorwba 8y agoThere's also this set of slides on the technology behind SeeInTheDark: http://graphics.stanford.edu/talks/seeinthedark-public-15sep16.key.pdf http://graphics.stanford.edu/talks/seeinthedark-public-15sep...
- trhway 8y agomultiple frames approach sounds similar to Hubble telescope approach.
- jessriedel 8y agoThe example images at your first link are much more informative than the OP link. Photo pairs should be compared after normalizing the total brightness of each image file. (Heck, I could take a single photo and display it at two different brightness settings and it would give basically same impression as the OP article to anyone not looking super carefully.)
- jjcm 8y agoThat video is a great demo. Very simple, very informative, and accurate from a data point.
- londons_explore 8y agoIt's notable that this 'accumulation' method effectively lets you have a near-infinite exposure time, as long as objects in the video frame are trackable (ie. there is sufficient light in each frame to see at least something). I'd be interested to see how night mode performs when objects in the frame are moving (it should work fine, since it will track the object), or changing (for example, turning pages of a book - I wouldn't expect it to work in that case).
- londons_explore 8y agoI would like super sensitive cameras like this to be used inside fridges to see the very faint glow of food going off. Chemical reactions by bacteria breaking down food produce light, enough for humans to see in only the darkest of places (if you live in a city, you won't ever encounter dark enough situations). A camera simulating a 1 hour exposure time in a closed refrigerator ought to be able to see it pretty easily.
- thestepafter 8y agoIt would be really interesting to then display this on the screen of smart refrigerators. Maybe you could even flag food that is above a certain brightness as spoiled.
- deleted 8y ago[deleted]
- Mahn 8y agoI never knew I wanted a "smart" fridge, but a fridge that can detect food going off sounds incredible. Sign me up if this is feasible.
- derefr 8y agoA fridge that prevents food going off would be better, no? That wouldn't require smarts, just replacing the air with pure nitrogen, like modern grain silos do. Doing it at "consumer scale" would probably only require the same sort of compressor that your fridge already uses for its cooling.
- viraptor 8y agoThere are some really low-tech things you can do to extend the life of things in the fridge. At least for vegetables, you can stick them in one of the fabric bags, add water, and they'll survive for weeks. And that costs just a few bucks. For meat we've got freezers. Is ready food going spoiled a big issue?
- 8y ago
- deleted 8y ago[deleted]
- lostmsu 8y agoAny hardware reason for it to only work on Pixel phones?
- shittyadmin 8y agoSupports a better Camera API - some people have cracked it to work on non-Pixel devices already though. See: https://www.celsoazevedo.com/files/android/google-camera/ https://www.celsoazevedo.com/files/android/google-camera/
- bscphil 8y agoCan you say more precisely what version you would need to get the new night stuff? I tried installing the latest "recommended" version on my Nexus 5x, but it refused to even install it.
- rdtsc 8y ago> Google says that its machine learning detects what objects are in the frame, and the camera is smart enough to know what color they are supposed to have. That is absolutely impressive. The color and text on the fire extinguishers along with the texture detail seen in the headphones in the last picture are just stunning. Congratulations to anyone who worked on this project!
- Johnny555 8y agoIt's impressive, but it also means that your camera isn't always going to capture what's there -- it'll capture what it guesses was there. I wonder how easily it is fooled to capture something that's not there?
- comboy 8y agoI think it's taking as many pictures as possible and using very slightly different angle to get as good resolution as possible with as little noise as possible (I have no idea if that's really what's happening here)
- Johnny555 8y agoThat's not what this article says, do you have reason to believe it's incorrect? The quote about object detection in the parent post came from the article.
- kortex 8y agoI'm sure the first step is taking many sharper short exposure shots (as opposed to longer exposures, which blur), then doing some tensor magic to stitch into a single image. Object detection alone won't give you sharp text in low light. You need a minimum number of photons hitting pixels.
- fla 8y agoFrom the article: > Although it’s not one single long exposure, Google’s night mode still gathers light over a period of a few seconds, and anything moving through the frame in that time will turn into a blur of motion.
- dhruvp 8y agoIf you're interested in learning more about this stuff the head of the Pixel team (Marc Levoy - Prof emeritus at Stanford) has an entire lecture series on this stuff from a class he ran at Google. They are here along with lecture notes:https://sites.google.com/site/marclevoylectures/home https://sites.google.com/site/marclevoylectures/home What's really cool is you can see him talk about a lot of these ideas well before they made it into the Pixel phone
- blt 8y agofrom what I can tell, Marc Levoy is the main reason why the Pixel has all these abilities.
- khazhou 8y agoLet's give credit to the entire engineering team.
- marclevoy 8y agoIndeed let's do that. At our weekly meeting yesterday, there were 18 engineers in the room. Every one of them is working hard to make this feature happen. My role at this point is mostly to kibitz, and help with the field testing.
- khazhou 8y agoWhat a fantastic reply. Much love, Marc.
- lawrenceyan 8y agoIf you're thinking of taking a look, I particularly enjoyed Lecture 7 of this series. The practical applications of Fourier analysis in image recognition/filtering/processing are really quite amazing. Plus, if you're at all curious about the technical details for how exactly something like Night Sight is implemented on the Pixel, understanding what Fourier transformations are and how they are utilized is vital.
- polskibus 8y agoHow does Pixel implementation of low light photography differ from Samsung? Are they comparable in photo quality?
- Kagerjay 8y agoWouldn't video still be extremely blurry? This is mostly for things not moving / pictures I wonder if this technology will eventually supercede military night vision goggles. Having the ability to add color perception at long distances could have useful for identifying things at night.
- celeritascelery 8y agoyou are right that this would be useless in its current state for video. But if it ever got smart enough to start stacking images on a 3d model of the world you could do some incredible stuff.
- Erwin 8y agoThe Huawei P20 shipped out in April with this feature -- I look forward to dxomark's analysis of the Pixel 3 phone compared to the P20, which currently remains on top: https://www.dxomark.com/category/mobile-reviews/ https://www.dxomark.com/category/mobile-reviews/ Upgrading from a 3-year old Samsung S6, where I could almost see the battery percentages drop off percent by percent, the P20 Pro's 4000 mAh battery has been great (too bad the wireless charging didn't appear until the new Mate P20 Pro).
- gwicks56 8y agoI'm hesitant to bring this up, for lots of different reasons, but people I know and mostly trust, who work in places where they should be in a position to know, all tell me the same thing: "Never buy Huawei anything, for any reason". Anybody have any real knowledge about this? @danielmicay is doing some interesting stuff in relation to being able to verify different Android build, but I have not tested yet to see.
- krackers 8y agoI'd assume it'd be in relation to some sort of China-related backdoor or datamining. About half a year ago intelligence agencies also publicly recommended against buying Huwawei/ZTE phones, for presumably whatever reasons the people you mentioned may be privy to.
- TheForumTroll 8y agoThe secret: They are not made by a US company. It is called "Propaganda" outside the US.
- 0x54D5 8y agoUnfortunately the P20 does not currently have a warranty in the United States.
- shaklee3 8y agoThe Huawei does not have this feature. Look at the review in your own link and you will see that the images for low light are comparable to the pixel or iPhone. If you compare night sight to those, it's completely different.
- golfer 8y agoWhy not use the actual article title? "Google’s Night Sight for Pixel phones will amaze you"
- jacksmith21006 8y agoThat title would sound positive and Google. That is not appropriate for HN.
- gpvos 8y ago"will amaze you" is an unnecessary clickbaity addition. They could have left Google in though.
- pavel_lishin 8y agoPrepare for a lot of cute sleeping baby photos on your feeds, folks. That's what I'll be using this for.
- nkoren 8y agoWhat the Pixel cameras are doing is staggeringly good. My father is the founder of https://www.imatest.com/ https://www.imatest.com/, and has a substantial collection of top-end cameras. He's probably in the top 0.0001% of image quality nerds. But most of the time, he's now entirely happy shooting on his Pixel.
- combatentropy 8y agoI enjoyed some of the articles on your father's other website, like http://www.normankoren.com/digital_tonality.html http://www.normankoren.com/digital_tonality.html, and hope he keeps it running.
- nkoren 8y agoYeah he's a good egg. I'm sure it'll keep going as long as he does. His startup actually grew out of those articles -- which were basically supposed to be his retirement hobby -- and now it employs 20 people! I'm really proud of what he's accomplished.
- vibrolax 8y agoThat's great news. I came into contact with your father's work with vacuum tube audio. I always admired his solid engineering approach without mysticism.
- PascLeRasc 8y agoWow, what a cool guy. I know we don't do AMAs on here but it'd be amazing to get HN to pick his brains for a bit.
- gingerbread-man 8y agoWe don't do them often, but there have been at least a couple of AMAs on here. Alan Kay https://news.ycombinator.com/item?id=11939851 https://news.ycombinator.com/item?id=11939851 Michael Siebel https://news.ycombinator.com/item?id=13895362 https://news.ycombinator.com/item?id=13895362 Ny AG Eric Schneiderman https://news.ycombinator.com/item?id=15853374 https://news.ycombinator.com/item?id=15853374 Scott Aaronson https://news.ycombinator.com/item?id=17425377 https://news.ycombinator.com/item?id=17425377 Sam Altman (he's done a few) https://news.ycombinator.com/item?id=12593689 https://news.ycombinator.com/item?id=12593689
- deleted 8y ago[deleted]
- bwang29 8y agoThe biggest challenge to do this technically is to use gyroscope data to work together with the stacking algorithm. It's hard to tune the gyro to work great for any phone. A pure software solution to analyze the perspectives transformation would be too slow.
- cma 8y agoThey probably have to have really well calibrated gyros for Daydream, with temperature calibration etc., unless the IMU/gyro is in the daydream holder itself.
- kartickv 8y agoIs it hard to tune the gyro because its accuracy is too low, or because it's accurate to begin with but degrades over the course of the long exposure? Is a pure software solution even reliable enough under these conditions? Slowness can be worked around by doing it in the background, and you get a notification when it's complete. Some people would be okay if that's the only way to get photos they wouldn't otherwise be able to get, short of buying an SLR.
- jakobegger 8y agoAll those shots look amazing, but they're of stationary objects. I really want to know how that works for people! 99% of photos I take are of people, and the lighting is always bad. Are there any photos of people?
- joshschreuder 8y agoThere's a photo of the author in the article.
- hammock 8y agoHow are you going to do a review of Night Sight and not even go outside? Every photo just taken in a room with the lights turned off. Come on, man. Tell your editor he needs to wait until nightfall.
- Yhippa 8y agoHuh? There are a bunch in the second half of the article.
- fuddle 8y agoThis reminds me of a similar project: "Learning to See in the Dark". They used a fully-convolutional network trained on short-exposure night-time images and corresponding long-exposure reference images. Their results look quite similar to the Pixel photos. http://cchen156.web.engr.illinois.edu/SID.html http://cchen156.web.engr.illinois.edu/SID.html
- endorphone 8y agoInteresting, but a tad rich with puffery. Pre-OIS Google did this with image stacking which was a ghetto version of a long exposure (stacking many short exposure photos, correcting the offsets via the gyro, was necessary to compensate for inevitable camera shake). There is nothing new or novel about image stacking or long exposures. What are they doing here? Most likely it's simply enabling OIS and enabling longer exposures than normal (note the smooth motion blur of moving objects, which is nothing more than a long exposure), and then doing noise removal. There are zero camera makers who are flipping their desks over this. It is usually a "pro" hidden feature because in the real world subjects move during long exposure and shooters are just unhappy with the result. The contrived hype around the Pixel's "computational photography" (which seems more incredible in theory than in the actual world) has reached an absurd level, and the astroturfing is just absurd.
- deleted 8y ago[deleted]
- windexh8er 8y agoWhile I'd like to believe pulling off these levels of photos is as easy as you proclaim (because I would have a lot of old shots that are usable) the level of discounting you assert here is unfortunate, because it's simply not true. The proof is in the reality that nothing has done this good of a job to date. If you do have examples that match or exceed please share and I will be in line to buy.
- endorphone 8y agoJust to be clear, I've been a serious photographer (amateur, not professional, but with all of the gear) for two decades, and actually made a pretty popular computational photography app for Android. I honestly don't know which part you're doubting. Long exposures? Do you doubt that other cameras can do long exposures? Do you doubt that they can do noise reduction? Do you doubt that OIS allows for hand-held long exposures, especially on wide-angle lenses? What are you doubting, because these are all trivial things that you can validate yourself. As to examples, you're wide-eyed taking a puff piece with some absolutely banal examples and exaggerated descriptions -- and zero comparable photos from other devices -- by someone who apparently knows very little about photography. How should I counter that? I can find millions of night streetscape photos that absolutely blow away the examples given. Generally if you're going to pander to a manufacturer, you at least talk about things like lux. In this case it's just "look, between this setting and that setting it's different, therefore no one else can do it".
- whoisjuan 8y agoDamn! That's honestly impressive! I started reading thinking it was going to be a simple brightness-up kind of thing, but it's incredible how they are able to recreate the whole photography based on an initial dark raw input. I must imagine that the sensor is doing an extra but un-perceptible long exposure than then is used to correct the lightning of the dark version.
- woolvalley 8y agoNow if we could only get this on APS-C & 1" compacts like the sony rx100 or fujifilm xf10. With first class smartphone integration and networking.
- bscphil 8y agoMy thoughts exactly. If a little IS work and some great noise reduction like they've shown here can look this good, can you even imagine what an APS-C or full frame could pull off?
- gingerbread-man 8y agoKind of a tangent, but it was really cool to see a picture of the author's Schiit Jotunheim headphone amp in the article. One of the founders wrote an amazing book on building a hardware startup: http://lucasbosch.de/schiit/jason-stoddard-shiit-happened-tablet-lblb.pdf http://lucasbosch.de/schiit/jason-stoddard-shiit-happened-ta....
- tjr225 8y agoThis might be a weird criticism but... making photos taken in the dark look like they are not actually dark seems kind of like a weird thing to do? I've struggled with my micro 4/3 camera to capture accurate night photographs, but the last thing I wanted of them was to be brighter than I was perceiving them to be. That said, the effect of some of these photographs is striking, and I'm sure the tech is interesting.
- Waterluvian 8y agoI hear what you're saying. But I almost always capture photos for data, not art, and would really benefit from this. As long as it is a setting, we both win.
- Splines 8y agoI think the idea is that you could capture a better photo with more detail up front then post-process afterwards to how you want it to look. Although I get what you're saying, some of the night photos looked great on the default camera.
- dannyw 8y agoDamn. I’ve had an iPhone since the 3G, but this is really tempting me to get a pixel.
- yanonymous2 8y agoThat's great, but we should find a different name for "photos" that change image information in the process.