9 ms·
Nvidia tool generates full 3D models from a single still image
- f0e4c2f7 5y agoWasn't there just a tool posted here to do this on a Show HN recently? I can't remember the name of it now. I believe you had to upload more than one image for it to work but they had an api. Edit: after looking this is what I was thinking of. usdz.app
- junon 5y agoYou might be thinking of Looking Glass.
- The_rationalist 5y agousually its NERFs derivatives
- amelius 5y agoObservation: stuff that used to come from SIGGRAPH now comes from NVidia.
- belter 5y agoThe paper: "Image GANs Meet Differentiable Rendering For Inverse Graphics And Interpretable 3D Neural Rendering" https://arxiv.org/pdf/2010.09125.pdf https://arxiv.org/pdf/2010.09125.pdf
- pixel_fcker 5y agoThat’s because nvidia hired like every graphics researcher.
- baybal2 5y agoMatroX GraphX circa 1999-2001 Matrox had a famous 2D to 3D demo
- xwdv 5y agoIt’s great for Indie game devs who just want quick assets and don’t want to go through the hassle of finding some artist and paying obscene money for a couple assets. My inability to get good consistent artwork is what has always demotivated me from actually finishing any kind of game.
- DaiPlusPlus 5y agoI remember as a kid messing around with early desktop-computer photogrammetry software: the kind where you have to print-out some black+white marker circles and place them around some trinket you want to capture in 3D. ..it worked, but the output *.3ds files were low-resolution and the textures only looked good from one direction - and the material's lighting was way off. So the concept is nothing new, but it's never as simple as taking even a bunch of photos, let alone 1 photo - there's so much information necessary for future renders that cannot be captured from photos alone.
- Grakel 5y agoMaybe art is worth its market rate.
- karmasimida 5y agoAs a demo .... I was expecting something more flashy than this one. The current one seems only OK-ish to me.
- vultour 5y agoI was worried about the legality of using real cars in your game. Then I noticed what it generated from the picture of a G-wagon.
- schleck8 5y agoAt this rate the metaverse might actually become a thing, if the hardware allows for enough immersion
- chillingeffect 5y agoIronically ttansforming K.I.T.T. revealed the limitations... but still excellent progress!
- ksec 5y agoApple has something similar, Object Capture. https://developer.apple.com/augmented-reality/object-capture/ https://developer.apple.com/augmented-reality/object-capture...
- cokeandpepsi 5y agototally different
- sydthrowaway 5y agoThis is photogrammetry
- saruberoz 5y agothis is really cool. ps: funny thing, seems like the tire all the same
- TheJoeMan 5y agoPlease let me know if this already exists, but: video games are becoming huge file sizes in part due to HD graphics. Why not just generate say the faces of pedestrians on the fly? Is it important that some character in my copy of the game looks identical in all copies of the game? I would argue it just needs a unique face to go with the name.
- TheDudeMan 5y agoIt can still be the same in every instance of the game -- generate from a known seed. This is basically good data compression, customized for faces.
- Liquix 5y agoYes! The tech mentioned in the article solves this problem: no more "we have to remove/compress all of these models/textures because our game can't be 300GB". Just use 10 high-res pictures of the object from all angles instead of a model. This could save orders of magnitude of disk space It also solves the problem of "we have to choose what objects to model because our 3D artists don't have infinite time". Now the artists review GAN output and tweak what the algorithm gets wrong, creating tons of content without needing to create every vertex from scratch. Very exciting
- picture 5y agoNot sure if "10 high-res pictures of the object from all angles instead of a model" saves any disk space at all considering models in games are usually just one high res picture uv wrapped to vertexes. Also, creating new content from GAN might not be super appealing for the artistic qualities of the game. Why not do what Escape from Tarkov seem to do with photogrammetry on real objects? Many props like soup cans, couches, whole environments, weapons, can be acquired in real world reasonably easily.
- Guest19023892 5y agoAre you referring to procedurally generated content? https://en.wikipedia.org/wiki/Procedural_generation https://en.wikipedia.org/wiki/Procedural_generation .kkrieger got a lot of attention for using this method and only being 96KB. https://www.youtube.com/watch?v=2NBG-sKFaB0 https://www.youtube.com/watch?v=2NBG-sKFaB0
- kristjansson 5y agoAstounding but I can't help be be underwhelmed at the results. Look at the Shelby Cobra dropped in at 1:01 [0] - it's a vaguely grey car-shaped thing like all of the rest of the models that captures little of the source image. [0]: https://youtu.be/gz5E9wszZSI?t=61 https://youtu.be/gz5E9wszZSI?t=61
- nacs 5y agoThe horse is hilariously bad: https://youtu.be/gz5E9wszZSI?t=53 https://youtu.be/gz5E9wszZSI?t=53
- lostcolony 5y agoYeah...that looks...frankly worse than the kind of thing I did in my undergrad 3d graphics class, where we did some edge detection, tessellated it, and projected it into 3D space.
- gbraad 5y agoKitt itself looks like a Nissan with a rounded butt :-)
- heleninboodler 5y agoIt seems like an April Fool's joke to me. The dramatic narration overlaid on lumpy terrible-looking cars made me laugh out loud several times. I mean, I'm sure what they're doing is technically impressive but the results are downright awful. All the cars look like they're beaten to hell and back and I honestly wouldn't be able to pick "KITT" out of a lineup of any other black car if it weren't for the cylon light on the front.
- sva_ 5y ago> The dramatic narration overlaid on lumpy terrible-looking cars made me laugh out loud several times. The horses did it for me. It made it look like some low effort parody.
- 5y ago
- fennecfoxen 5y agoKITT is okay, but I prefer William Daniels as John Adams (or as Mr. Feeney).
- Dracophoenix 5y agoHard to believe that Mr. Daniels is still alive all these years later.
- Splizard 5y agoI think it's degrading to call this a 'tool', it's an intelligence.
- picture 5y ago"Intelligence" seems kinda hype-y to me. Machine learning is mainly just statistics. We still have just a ways to go until real "intelligence" afaik
- visarga 5y agoYeah, much of human intelligence is also just statistics. We do a lot of unsupervised learning and reasoning based on correlation. The opposite of this would be causal reasoning but that is hard for humans as well. Not everyone can use causal reasoning, it requires many years of training. The discovery of causal mechanisms (science) is slow and it takes many people to do it. For example the president of Turkey believes interest rates should be lowered in a hyperinflation. What can we do? Causal reasoning is hard. I think we do something else, slightly different than causal reasoning. We're generating hypothesis and testing them out, in an iterative process. Like a GAN (generator+discriminator) or Actor Critic (policy + value function). More famously, AlphaGo was generating many rollouts for each move, one model to generate moves, another to evaluate their consequences.
- smrtinsert 5y agoIs this KITT appearance considered canon?
- exodust 5y agoThey turned KITT into a blurry 3d blob, poorly lit to hide the flaws. Barely qualifying as a full 3d model. "Black car" is the appropriate name for the result, nothing more. They disabled comments on the article, and on the youtube video. Combined with YouTube's newly hidden dislike ratings, we can't know how many viewers are unimpressed. Hype has more breathing room than ever before.
- monkeydust 5y agoCurious, does anyone know software that could generate body sizing for an individual based in image stills or video taken from a normal mobile phone? Essentially creating a 3d human model but with high degree of accuracy.
- GistNoesis 5y agoI don't know of one yet. Lot of people are working on it, using various strategies. The grail being able to provide a virtual fitting-room experience. Or monitoring fitness of individuals. A 2021 paper with code (I haven't tried it): https://github.com/akashsengupta1997/hierarchicalprobabilistic3dhuman https://github.com/akashsengupta1997/hierarchicalprobabilist... to generate a 3d morph-able human-shape in the correct pose from a single picture. A picture always has scale uncertainty (a 2-meter human viewed from 1 meter away look the same that a 1-meter human viewed from 0.5 meter away), so that is an additional problem that must be taken care of. But now recent phone have 3d sensor that provide information that could be useful. To generate 3d human models there is also makehuman . In the old days there was a soft called facegen, that could generate 3d face models and could automatically fit their parameters to two pictures using an iterating refinement procedure. Deep-learning usually estimate everything jointly in a single step so they are faster, but often less accurate. But there exist models that learn to refine a previously generated model, so you can apply them repeatedly and get improved quality (Denoising Diffusion Probabilistic Models is one generic class of models that does this).
- anaisbetts 5y agoA commercial (and therefore, far more expensive) version of this tool is https://www.capturingreality.com https://www.capturingreality.com. Cool that nVidia is starting to make this more accessible, even if the results aren't quite usable yet
- alophawen 5y agoThis is cool but this also not how any horse back ever looked: https://www.youtube.com/watch?v=gz5E9wszZSI&t=53s https://www.youtube.com/watch?v=gz5E9wszZSI&t=53s
- cinntaile 5y agoI don't get the negative comments. The results look alright, but they lack detail at this point. This will improve over time and there are probably a lot of applications that don't need this level of detail anyway. I'd guess this will put some 3D modelers out of a jobb x years into the future, classic disruptive tech stuff!
- tsumnia 5y agoI like to take a line from Two Minute Papers[1] - If this is what one "paper" was able to produce, imagine the improvements will be two "papers" down the line. [1] https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg
- dtgriscom 5y agoSame. These results are far better than I (with little 3D modeling experience) could generate. A pro could do better, but a) do you have a pro available? and b) can they afford the time to model all of the item you want to throw into your mockup?