9 ms·
Apple releases open-source model that instantly turns 2D photos into 3D views
- dmos62 10mo agoAnyone's aware of something similar for making interactive (or video) tours of apartments from photos?
- neom 10mo agoExamples: https://apple.github.io/ml-sharp/ https://apple.github.io/ml-sharp/ Paper: https://arxiv.org/abs/2512.10685 https://arxiv.org/abs/2512.10685
- the8472 10mo agoimo https://x.com/SadlyItsBradley/status/2001227141300494550 https://x.com/SadlyItsBradley/status/2001227141300494550 is a better demo than their own project page
- cromulent 10mo agoPrevious discussion: https://news.ycombinator.com/item?id=46284658 https://news.ycombinator.com/item?id=46284658
- b112 10mo agoAh great. Easier for real estate agents to show slow panning around a room, with lame music. I guess there are other uses?? But this is just more abstracted reality. It will be innacurate just as summaried text is, and future peoples will again have no idea as to reality.
- tim1994 10mo agoFor panning you don't need a 3D view/reconstruction. This also allows translational camera movements, but only for nearby views. Maybe I am overly pedantic here, but for HN I guess thats appropriate :D
- parpfish 10mo agoFor a good slow pan, you don’t need 3d reconstruction but you DO need “Ashokan Farewell”
- stevep98 10mo agoIt will be used for spatial content, for viewing in Apple Vision Pro headset. In fact you can already turn any photo into spatial content. I’m not sure if it’s using this algorithm or something else. It’s nice to view holiday photos with spatial view … it feels like you’re there again. Same with looking at photos of deceased friends and family.
- bertili 10mo agoHN discussion 11 days ago: https://news.ycombinator.com/item?id=46284658 https://news.ycombinator.com/item?id=46284658
- dang 10mo agoThanks! Macroexpanded: SHARP, an approach to photorealistic view synthesis from a single image - https://news.ycombinator.com/item?id=46284658 https://news.ycombinator.com/item?id=46284658 - Dec 2025 (108 comments)
- transcriptase 10mo agoI love how virtually no GitHub instructions related to AI simply work as written. Each assumes you already have their developer environment configured to have the tool work, but simply don’t have it compiled yet.
- personjerry 10mo agoThis repo's instructions seem to work as written just fine?
- gjsman-1000 10mo agoIs this the same model as the “Spatial Scenes” feature in iOS 26? If so, it’s been wildly impressive.
- mercwear 10mo agoI am thinking the same thing, and I do love the effect in iOS26
- alexford1987 10mo agoIt seems like it, although the shipped feature doesn’t allow for as much freedom of movement as the demos linked here (which makes sense as a product decision because I assume the farther you stretch it the more likely it is to do something that breaks the illusion) The “scenes” from that feature are especially good for use as lock screen backgrounds
- nyc_pizzadev 10mo agoYa, I like when it’s automatically done on my featured photo, gives the phone a very 3D look and feel.
- basisword 10mo agoI assume this is the same spatial scenes feature that was on visionOS prior to OS 26. In my experience that was really incredible. You could take a standard 2D photo of someone and suddenly you were back in the room with them.
- jtrn 10mo agoI was thinking of testing it, but I have an irrational hatred for Conda.
- moron4hire 10mo agoYou aren't being irrational.
- nativeit 10mo agoPerhaps they lived outside of the kingdom, with an evil Stepmother who moved very slow, struggled with complex dependency collisions, and took up a bunch of unnecessary space? Such an experience could leave one very traumatized towards Conda, even though their real problems are the unresolved issues with their stepmother…
- deleted 10mo ago[deleted]
- jtreminio 10mo agoYou can simply use a `uv` env instead?
- optionalsquid 10mo agoYou could use pixi instead, as a much nicer/saner alternative to conda: https://pixi.sh https://pixi.sh Though in this particular case, you don't even need conda. You just need python 3.13 and a virtual environment. If you have uv installed, then it's even easier: git clone https://github.com/apple/ml-sharp.git cd ml-sharp uv sync uv run sharp
- jtrn 10mo agoThe hate is so irrational I can’t stop feeling that any project that even uses Conda HAS to be terrible. Like a chef that creates a recipe with shit as an ingredient. I could exchange the shit for sugar, but why bother, the chef is obviously insane. I’m really sorry if anyone that worked in this ever reads this. But Conda is just triggering me.
- RobotToaster 10mo agohttps://raw.githubusercontent.com/apple/ml-sharp/refs/heads/main/LICENSE_MODEL https://raw.githubusercontent.com/apple/ml-sharp/refs/heads/... "Exclusively for research purposes" so not actually open source.
- hwers 10mo ago[flagged]
- tremon 10mo agohttps://opensource.org/osd#fields-of-endeavor https://opensource.org/osd#fields-of-endeavor > The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, [..]
- ecb_penguin 10mo agoWhile most people follow the OSD criteria, there is nothing that says open source software must follow it. Nor is the OSD the only set of criteria or the only definition. Open source means the source is available. Anything else is just political.
- nativeit 10mo agoNow you get to graduate into the pedantry of defining the word “source”.
- ecb_penguin 10mo agoThere's no reason for pedantry. Source is pretty well defined.
- immibis 10mo agoWhich is how we know that AI models don't have any - and therefore can't be open source.
- d_watt 10mo agoI’ve been using some time off to explore the space and related projects StereoCrafter and GeometryCrafter are fascinating. Applying this to video adds a temporal consistency angle that makes it way harder and compute intensive, but I’ve “spatialized” some old home videos from the Korean War and it works surprisingly well. https://github.com/TencentARC/StereoCrafter https://github.com/TencentARC/StereoCrafter https://github.com/TencentARC/GeometryCrafter https://github.com/TencentARC/GeometryCrafter
- deleted 10mo ago[deleted]
- jokoon 10mo agodoes it make a mesh? doesn't seem very accurate, no idea of the result with a photo of large scene, that could be useful for level designers
- wahnfrieden 10mo agoNo
- andybak 10mo agoGaussian splats
- avaer 10mo agoIt doesn't but it's pretty trivial to do if all you want is a pinholed mesh. I managed to one-shot it by mixing in the mesh exporter from https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror but at that point you might as well use HWM, which is slower but much better suited to the level design use case. Note that the results might not be as good as you expect, because this does not do any angled inpainting -- any deviation from the camera origin and your mesh will be either full of holes or warped (depending on how you handle triangle rejection) unless you layer on other techniques far outside the scope of this model. And note that although HWM itself does support things like multi-image merging (which ml-sharp does not), in my testing it makes so many mistakes as to be close to useless today. If you want something different that is designed for levels, check out Marble by World Labs.
- Invictus0 10mo agoApple is not a serious company if they can't even spin up a simple frontend for their AI innovations. I should not have to install anything to test this.
- consonaut 10mo agoIt's included in the ios photo gallery. I think this is a separate release of the tech underneath.
- londons_explore 10mo agoWhat user feature does it power?
- givinguflac 10mo agoLiterally what this model does- create seemingly 3d scenes from 2d images, in the iOS photos app. It works even better when you take a real spatial image, which uses dual lenses.
- avaer 10mo agoThis is a free research project on GitHub. I think I'd rather apple focus on making hardware than hoarding GPUs for PR stunts to prove they are a "serious company".
- victormustar 10mo agoHugging Face model: https://huggingface.co/apple/Sharp https://huggingface.co/apple/Sharp and demo: https://huggingface.co/spaces/ronedgecomb/ml-sharp https://huggingface.co/spaces/ronedgecomb/ml-sharp
- nativeit 10mo agoWeird how “hugging face” is a heartwarming little smiley face, while “face hugger” is a terrifying alien xenomorph. Seems like there’s an analogy to be made there…
- lvl155 10mo agoI don’t know when Apple turned evil but hard for me to support them further after nearly four decades. Everything they do now is directly opposite of what they stood for in the past.
- tsunamifury 10mo agoApple absolute Never believed in open source in the past so yes. They are not the same
- lostlogin 10mo agoWhere does Swift fit into this? I haven’t followed along but believe it’s open source and a search appears to confirm this?
- lvl155 10mo agoSwift language is open source but the entire ecosystem is as closed as they get. The fact that no one is building anything outside of the ecosystem says everything about Swift and Apple’s intent. The fact that they still won’t support Linux on M chips also says they don’t care.
- knorker 10mo agoNot never. Woz championed some of that in the 1970s. It's before my time, but the Apple II was pretty open as I understand it.
- saagarjha 10mo agoCurious what this has to do with the post?
- lvl155 10mo agoApple trying to “open-source” something is pretty relevant. I don’t trust them at all. People constantly go at Microsoft but what Apple has done in the last 15 years is far worse. Their monopolies have had far worse impact than whatever Microsoft ever did with Windows and IE.
- burnt-resistor 10mo agoDamn. I recall UC Davis was working on this sort of problem for CCTV footage 20 years ago, but this is really freakin' progress now.
- hermitcrab 10mo ago"Sharp Monocular View Synthesis in Less Than a Second" "Less than a second" is not "instantly".
- ethmarks 10mo agoWhat would your definition of "instantly" be? I would argue that, compared to taking minutes or hours, taking less than a second is fast enough to be considered "instant" in the colloquial definition. I'll concede that it's not "instant" in the literal definition, but nothing is (because of the principle of locality).
- cubefox 10mo agoWittgenstein, Philosophical Investigations, §88: > (...) Now, if I tell someone: "You should come to dinner more punctually; you know it begins at one o'clock exactly"—is there really no question of exactness here? because it is possible to say: "Think of the determination of time in the laboratory or the observatory; there you see what 'exactness' means"? "Inexact" is really a reproach, and "exact" is praise. (...)
- 0_____0 10mo agoIf you're concerned by that, I have some bad news about instant noodles.
- reaperducer 10mo agoFolgers on line one.
- analog31 10mo agoI wonder if it helps that a lot of people take more than one picture of the same thing, thus providing them with effectively stereoscopic images.
- Coneylake 10mo agoAlso, frames from live photos
- bbstats 10mo agowould love a multi-image version of this.
- pcurve 10mo ago[flagged]
- saagarjha 10mo ago1. People with foreign sounding names may have been born in the United States. 2. People who were born outside the United States but moved here to do research a while back don’t suddenly stop doing research here.
- foota 10mo agoI'm not trying to be too pc, but you can't really tell based on someone's name where they were born. That said, the US only has some 5% of the worlds population (albeit probably a larger proportion of the literate population), so you'd only expect some fraction of the world's researchers to be US born. Not to mention that US born is an even smaller fraction of births (2.5-3%, by Google), so you'd expect an even smaller fraction of US born researchers. So even if we assume that we're on par with peer countries, you'd only expect US born researchers to be a fraction of the overall research population. We'd have to be vastly better at educating people to do otherwise, which is a longshot. Obviously this makes turning away international students incredibly stupid, but what are we to do against stupidity?
- chairhairair 10mo agoHow do you know where the authors were born?
- raphman 10mo agoFWIW, many of the researchers on the paper did not study in the U.S. but immigrated after their PhD studies. I checked the first, middle, and last author: Lars Mescheder got his PhD in Germany, Bruno Lecouat got his PhD in France, Vladlen Koltun got his PhD in Israel. (Edit: or maybe they did not actually immigrate but work remote and/or in Europe)
- _fizz_buzz_ 10mo agoApple is also a global company and has offices and research labs world wide. At least a couple of the authors seem to work for Apple but at their German lab.
- deleted 10mo ago[deleted]
- chmod775 10mo agoBig day for VR pornography! I'm not kidding. That's going to be >80% of the images/videos synthesized with this.
- rcarmo 10mo agoGives the term "Gaussian splat" an entirely different meaning...
- wwwlouishinofun 10mo agoNow I know what Gaussian scattering is.
- avaer 10mo agoUnfortunately not as significant as you'd think. The output is not automatically metrically scaled (though you can use postprocessing to fix this, it's not part of this model). And you can't really move around much without getting glitches, because it only inferences in one axis. It's also hard capped at 768 pixels + 2 layers. Besides depth/splatting models have been around for quite a while before this. The main thing this model innovates on is inference speed, but VR porn isn't a use case that really benefits from faster image/video processing, especially since it's still not realtime. This year has seen a lot of innovation in this space, but it's coming from other image editing and video models.
- chmod775 10mo agoIt's not for moving around, but for turning some image into a stereoscopic one (or 2 side-by-side images if you will). Lots of techniques for this exist, which usually turn an image into depth information using AI and then use any number of approaches to generate/warp 2 offset images from it. So far the best looking results are still achieved with good old mesh warping and no inpainting at all. This may change that.
- avaer 10mo agoAh, but if we're not talking 6DOF what's new with ml-sharp? We've had good autostereoscopy for a couple of years at least. > So far the best looking results are still achieved with good old mesh warping and no inpainting at all. I agree > This may change that. Seems not to be the case in my testing. The splats are too fine and sparse to yield an improvement. There are actually better (slower) image -> splat models than ml-sharp (with much higher dynamic range for the covariance) but I still don't use them over meshes for this. The only improvements ml-sharp seems to add to the SOTA is 1) speed and 2) an interesting 2-focal layer architecture, but these are somewhat tangential steps.
- rcarmo 10mo agoThis is a dupe. A couple of weeks ago I forked it and got the rendering to work in MPS: https://github.com/rcarmo/ml-sharp https://github.com/rcarmo/ml-sharp
- malshe 10mo agoThanks! This looks nice
- coffeecoders 10mo agoI feel like being in a time loop. Every time a big company releases a model, we debate the definition of open source instead of asking what actually matters. Apple clearly wants the upside of academic credibility without giving away commercial optionality, which isn't unsurprising. Additionally, we might need better categories. With software, flow is clear (source, build and binary) but with AI/ML, the actual source is an unshippable mix of data, infra and time, and weights can be both product and artifacts.
- ericflo 10mo agoI don't think it isn't unsurprising :)
- jama211 10mo agoWait so you are surprised?
- basisword 10mo agoI'm glad you said it. Incredible tech and the top comment is debating licensing. The demos I've seen of this are incredible and it'll be great taking old photos (that weren't shot with a 'spatial' camera) and experiencing them in VR. I think it sums up the Apple approach to this stuff (actually impacting peoples lives in a positive way) vs the typically techie attitude.
- mabedan 10mo ago> which isn't unsurprising There has to be an easier combination of words for conveying the same thing.
- darig 10mo ago[dead]
- backtogeek 10mo agoLicense arguments aside, pretty cool.
- ww520 10mo agoIs the model in ONNX format or PyTorch format?
- vednig 10mo agofacebook worked on a similar project almost 5 years back
- yalogin 10mo agoIs this already integrated into the latest iOS? If so it’s not good. It only works on a few images and for the most part the rendering feels fake and somehow incoherent
- RickyLahey 10mo agono
- bdelmas 10mo agoI’m so sad I had this idea at least 6 years ago but I didn’t have the connections to make it happen. But that’s nice that they released the project. Apple open sourcing their tech?