8 ms·
Facebook Now Lets You Turn Any 2D Photo into a 3D Image Using AI
- gpu_explorer 7y agoThey can make this very cool yes, but still isn't enough for me to enter Facebook. They should have more money for privacy.
- vidanay 7y agoWhere "3D image" = amorphous blob.
- chrisco255 7y agoI really want background replacement to be totally seamless. We've got to be close to that with this technology.
- mcphage 7y agoThe picture of Trevi Fountain has a very complicated 3D shape, almost none of which was inferred correctly.
- sk0g 7y ago3D Image ≈ Lovecraftian monstrosity. I've seen a few examples of this, and the results are mixed in a rather entertaining way!
- raxxorrax 7y agoTo be fair, raw data from "real" stereoscopic approaches looks fairly similar. Definitely a good result, but certainly something I have seen before in a comparable quality. Hard to judge from the examples though.
- m3kw9 7y agoI would call that 2.1D the .1 adds parallelax effects
- dheera 7y agoIt's 3D depth estimation from a 2D image. There is no such thing as 2.1D, 2.5D unless you are either a salesperson or a theroetical mathematician. Parallax effects are not 1/10 of a dimension in any sense of the word.
- drdeca 7y agoObviously it does not literally add 1/10 of a dimension. However, saying "2.1 dimensional" gives people the idea of "oh, it is still mostly the same as a 2 dimensional thing, but is a little bit closer (in some sense) to being 3 dimensional.".
- deleted 7y ago[deleted]
- sarakayakomzin 7y ago>There is no such thing as 2.1D, 2.5D unless you are either a salesperson or a theroetical mathematician. there also wasn't a Lion King 2.5 until there was.
- adwi 7y agoIn commercial motion graphics/animation, 2D images animated in 3D spade is widely referred to as “2.5D”
- justlexi93 7y agoBut the photos that you will send via messenger or upload lost its quality.
- saagarjha 7y agoI wonder if this could be used to improve background removal tools.
- Mr_P 7y agoResearchers at Adobe published similar work several months ago: https://news.ycombinator.com/item?id=20978055 https://news.ycombinator.com/item?id=20978055 Interestingly, FB folks used a vastly simpler network architecture, but it probably still works because they have access to so much real data. In contrast, the Adobe paper had a fairly involved network with pretrained VGG features, possibly to compensate for only having synthetic data to train on.
- dheera 7y agoIt's also worth noting that FB doesn't need a lot of accuracy in the depth data for this to be "good enough" for a simple wobble effect on a phone. It definitely doesn't need to be held to the same standards that would be held for a monocular depth estimator for, say, an autonomous vehicle or robot; in FB's case it doesn't matter whether the tree is 4m or 6m away, and really only matters that it's roughly behind the dog or cat or whatever. Separately, depth estimators like this are also useful to fake the depth of field effects of large-aperture lenses, which I'm pretty sure is used on the latest Pixel and iPhone in their background-blurring "Portrait" mode; Facebook could quite possibly include that as an in-app feature to make such effects accessible to everyone with cheaper phones as well.
- spectramax 7y agoHow many people like watching 3D movies at home or at a movie theater? Sometimes old fashioned 2D images are just fine. The content, story and artistic aspects of the image is what matters, not some cool half-baked 3d trick that is... frankly a distraction. I don't mean to criticize the research and engineering that went into doing something like this - it truly remarkable. I refer to the application possibilities that we might see in the future - imagine the web full of these 3d images, I think that would be a pretty noisy and distracting place.
- numpad0 7y agoI think the 3D effect is just a conversation starter. The purpose of this is to add couple new slider options to Lightroom.
- netsharc 7y agoIf you extrapolate where social media apps are taking us, first it was to "keep in touch" (however superficial) with people, even when it's a tweet that "they're pooping" or an IG of what they're having for lunch. And then came Periscope livestreams and Snapchat stories (stolen by Instagram and Facebook), so you can see a live video of what your friends are doing. FB bought Oculus, I wonder if their fantasy of the future is for realtime immersive sharing of experiences, your friend is on top of the Eiffel tower, you put on your goggles, and boom, you're there with them with a 360 degree video. At least that's the technological side, personally I think this simulacra of friendship/socializing is terrible.
- bryan_w 7y agoYou should watch some of sucks F8 keynotes. He lays out basically what you're saying
- tehsauce 7y ago"convoluted neural network" :)
- simon_000666 7y agoCan anyone recommend a good open source version of this?
- mih 7y agoNever tried it myself, but this might be an alternative - https://github.com/sniklaus/3d-ken-burns https://github.com/sniklaus/3d-ken-burns
- folli 7y agoWhy not directly link https://ai.facebook.com/blog/-powered-by-ai-turning-any-2d-photo-into-3d-using-convolutional-neural-nets/ https://ai.facebook.com/blog/-powered-by-ai-turning-any-2d-p...
- reportgunner 7y agoWhy did we need this ? Are we now going to remake all pictures into """3D""" and leave the boring 2D pictures in dust ?
- illwrks 7y agoAnyone remember an iPhone app canned Seene (I think) that did similar about 7 years ago? It lacked the AI aspect but essentially you took several photos from different angles and it created some form of depth map with the central image as a texture (if I remember correctly) I was amazed by this at the time.
- fsloth 7y agoPhotogrammetry is a commoditized set of algorithms and workflows to do this. https://en.wikipedia.org/wiki/Photogrammetry https://en.wikipedia.org/wiki/Photogrammetry There is a good introduction to production use in this GDC presentation from DICE: https://www.gdcvault.com/play/1023272/Photogrammetry-and-Star-Wars-Battlefront https://www.gdcvault.com/play/1023272/Photogrammetry-and-Sta... Examples of applications that are usable for individuals include: https://www.3dflow.net/3df-zephyr-pro-3d-models-from-photos/ https://www.3dflow.net/3df-zephyr-pro-3d-models-from-photos/ https://www.agisoft.com/ https://www.agisoft.com/ https://www.capturingreality.com/ https://www.capturingreality.com/ And free/open source/academic: https://colmap.github.io/ https://colmap.github.io/ http://ccwu.me/vsfm/ http://ccwu.me/vsfm/ https://alicevision.org/ https://alicevision.org/
- zimpenfish 7y agohttps://get.display.land https://get.display.land is also pretty good. It did a passable job of our local statue[1] in about 3 minutes of capturing + [some time of their cloud processing]. (Now it's not in the same ballpark as the capture I made with 5 minutes of taking ~90 photos and the ~20 hours of Meshroom processing but that's understandable.) [1] https://www.display.land/s/4QeZc8gT77beNLx44OdvDT https://www.display.land/s/4QeZc8gT77beNLx44OdvDT
- deleted 7y ago[deleted]
- burntoutfire 7y agoThe difference here is that they (FB) are able to create 3D model based on only one photo. If you have multiple photos from different angles, you can just use geometric relationships to figure out where the points lie in 3D space. However, with only one picture, you need to "guess" (using ML), as there's not enough data for proper 3D reconstruction.
- etiam 7y agoI don't need Facebook's permission to do that.
- Hoasi 7y agoHey! Here is a novel way to get people to submit their photos to Facebook.
- dTal 7y ago>If you’re more interested in how this impacts you, just know this: whether you want to convert “decades-old family photos” to 3D, or you want to convert your professional portraits into 3D-like creations, FB now lets you do that. You just need to upload it using the Facebook app on “an iPhone 7 or higher, or any recent midrange or better Android device.” ...aaaand that's the motivation for this feature - persuading people to upload more photos to Facebook.
- xeromal 7y agolol, you say this like it's a revelation. Why would a company build a feature in their product and not expect people to use their product to use the feature.
- ThePadawan 7y agoThere's a significant difference between "this works for all your current pictures on Facebook" and "you need to choose to upload some specific pictures on this specific app to use this feature". The latter will get you a far better information on what people want to see turned into 3D.
- IAmEveryone 7y agoThe text is ambiguous, but I would interpret it as "Photos need to be on Facebook to do this, so those not already on[...]" rather than "This works only on Photos newly uploaded".
- shawnz 7y agoWhy would they need to restrict the feature to new photos only to get that information? They could just use telemetry and see what photos people pick if they wanted to.
- dillonmckay 7y agoLike pytorch?
- peteretep 7y ago
- fasicle 7y agoWould this work for converting 2D panorama photos to 3D for VR purposes, essentially creating a fake stereoscopic photo?
- TeMPOraL 7y agoI remember playing with effects like this ~8 years ago. My first instinct would be to laugh at the "AI" part, but I'm guessing the point of the exercise isn't the parallax effect - the point is that they've actually managed to get a neural network to do that.
- IAmEveryone 7y agoI think the point is that it's far better than anything you may have been doing 8 years ago?