9 ms·
Generative Models: What do they know? Do they know things? Let's find out
- dkarras 3y agoThis is pretty remarkable. So these really do learn humanly interpretable representations and not only doing some magic in the billion dimensional hyperplane that we can't hope of deciphering.
- corysama 3y agoAs an old 3D graphics engineer, the fact that albedo is in there is as just striking as it should be expected. The core components of physically based rendering are position (derivable from image XY and depth), surface normal, incoming light, and at least albedo + one of a few variations on surface material properties such as specularity and roughness. That the AI is modeling depth is pretty expected. Modeling surface normal is a nice local convolution of depth. But, modeling albedo separated from incoming light is great. I wonder if specularity is hiding in there too.
- hackerlight 3y agoIt's a good depth map, too. Better than other tools I've seen which require fiddling with lots of knobs to get a good result. Might be useful for textures for parallax mapping.
- viraptor 3y agoI'm mostly ok with the idea that the text-to-image models do this. What blows me away is that the autoregressive model does it too. We're not even asking it to produce a specific object which has a meaning or context assigned. The network learned to produce those representations as just "this is a useful decomposition if I want to recreate the same thing again". The jump from image compression to figuring out how to represent objects in space is insane. And makes me want to feed some optical illusions through that network - what's the depth map of the infinite staircase?
- theygotit 3y ago[dead]
- int_19h 3y agoI find it amazing that, for all the evidence we have of generative models having some fairly complex internal model of the world, people still insist that they are mere "stochastic parrots" who "don't really understand anything".
- mewpmewp2 3y agoFrom the other side however if you really think about it, our understanding of everything must be stochastic as well. So perhaps this sort of thing yields in many complexities that we are not aware of. How would I know I am not a stochaistic parrot of some sort. I am just neural nets traine on current envrionment while the base model that is dependent on DNA, through evolution and natural selection of the fittest. Same as currently competing LLMs where the best one will win out.
- int_19h 3y agoYou're not wrong, but the "stochastic parrot" claim always comes with another, implicit one that we are not like that; that there's some fundamental difference, somehow, even if it is not externally observable. Chinese room etc. In short, it's the good old religious debate about souls, just wrapped in techno-philosophical trappings.
- cthalupa 3y agoYou could leverage the exact same accusation against the other side - we know fundamentally how the math works on these things, yet somehow throw enough parameters at them and eventually there's some undefined emergent behavior that results in something more. What that something more is is even less defined with even fewer theories as to what it is than there are around the woo and mysticism of human intelligence. And as LarsDu88 points out in a separate thread, there are alternative explanations for what we're seeing here besides "We've created some sort of weird internal 3D engine that the diffusion models use for generating stuff," which also meshes closely with the fact that generations routinely have multiple perspectives and other errors that wouldn't exist if they modeled the world some people are suggesting. If there's something more going on here, we're going to need some real explanations instead of things that can be explained multiple other ways before I'm going to take it seriously, at least.
- goryramsy 3y agoSSL error? Just me?
- corysama 3y agoWorks on my machine. Alternative link: https://github.com/duxiaodan/intrinsic-lora https://github.com/duxiaodan/intrinsic-lora
- 01HNNWZ0MV43FF 3y agoWorks here. "Verified by DigiCert". SHA-256 fingerprint of 38:2C:D4:2D:33:C0:2B:C6:67:8E:65:7C:E1:7B:84:6D:04:73:A7:E7:91:CD:B3:5B:8E:AD:90:1A:F1:E1:1A:08
- bbor 3y agoI have no idea what Toyota or adobe are up to and why they’re funding research with a name like this, but I fucking love it. It’s science, let’s get some whimsy back in here!! More materially: Optimized with a small set of labeled images, our model-agnostic approach adapts to various generative architectures, including Diffusion models, GANs, and Autoregressive models. Am I correct in understanding that this is purely a visuospatial tool, and the examples aren’t just visual by coincidence? Like, there’s no way to stretch this to text models? Very new to this interpretability approach, very impressive.
- raincole 3y agoBojack Horseman reference that we didn't know we need.
- seattle_spring 3y agoWhat is this, a crossover episode?
- sp332 3y agoThere's research into editing facts in language models. https://rome.baulab.info/ https://rome.baulab.info/
- frri 3y agoSi hzux
- blovescoffee 3y agoYou have no idea why Toyota or Adobe are funding computer vision research?
- bbor 3y agolol yeah a little. A) it said something very odd sounding like “Toyota university of Chicago”, which wtf why does Toyota have a university, and B) most labs would be hesitant to publish a paper with an extraneous clause in the title just to reference an absurd cartoon
- Donberry 3y ago[dead]
- dougmwne 3y agoThis is awesome! So one of the big reasons there was hype about Sora is that it felt very likely from watching a few videos that there was an internal physical simulation of the world happening and the video was more like a camera recording that physical and 3D scene simulation. It was just a sort of naïve sense that there HAD to be more going on behind the scenes than gluing bits of other videos together. This is evidence, and it’s appearing even in still image generators. The models essentially learn how to render a 3D scene and take a picture of it. That’s incredible considering that we weren’t trying to create a 3D engine, we just threw a bunch of images at some linear algebra and optimized. Out popped a world simulator.
- whimsicalism 3y ago> just a sort of naïve sense that there HAD to be more going on behind the scenes than gluing bits of other videos together People really still think that's all thats happening?
- 0_____0 3y agoWho is people? Most folks only vaguely know about generative AI if they know about it at all. I'm technical but not in software specifically and usually ignore most AI news, so GP's comment is in fact news to me!
- sigmoid10 3y agoThere's a lot of nonsensical babble out there from people who have minimal technical insight - both on the sceptic and on the enthusiast side of AI. It really wasn't clear for the longest time how these models generate things so well. Articles like this one are still rare and comparatively new. And they certainly haven't been around when less informed enthusiasts where already heralding AGI. On the other hand, we have accrued quite a bit of evidence by now that these models do far more than glue together training data. But there are still sceptics out there who spread this sort misinformation.
- beepbooptheory 3y ago
- ukuina 3y agoThe name is a reference to a fictional gameshow in Bojack Horseman called "Hollywoo Stars and Celebrities: What Do They Know? Do They Know Things?? Let's Find Out!" https://bojackhorseman.fandom.com/wiki/Hollywoo_Stars_and_Celebrities:_What_Do_They_Know%3F_Do_They_Know_Things%3F%3F_Let%27s_Find_Out https://bojackhorseman.fandom.com/wiki/Hollywoo_Stars_and_Ce...!
- echelon 3y agoI adore that show so much. I have this sticker on my laptop. If you haven't seen Bojack Horseman, it's funny and heartfelt. Palpably existential. If that kind of thing speaks to you, you owe yourself a watch. In terms of a complete animation package, I think it easy outdoes Futurama. There's so much relatable depth to it. It hits hard, but it stays lighthearted enough to make you feel good about it. As it turns out, I'm now working on filmtech, so that "Hollywoo" sticker fits me even better now.
- wudfar 3y ago[dead]
- the_af 3y agoI want to like Bojack and I know it has drama and heartfelt moments (watched all of season 1 I think), but in my opinion it's undone by its moments of "wacky" humor. I don't even mean that there are humanoid animals and humans living together, no explanation -- I can embrace that. I mean the Simpsons/Family Guy kind of humor... I could do without it and I think it would make the show better. Or maybe it does get better after season 1? Since everyone seems to love it.
- KerrAvon 3y agoIt’s very Los Angeles nihilist in tone. If that doesn’t make you happy, maybe skip it.
- yosame 3y agoSeason 1 is definitely the weakest season, especially at the start. There's definitely still wacky humour in the rest of the show, but it loses the family guy cutaway gags. I'd personally recommend giving season 2 a shot, if you still don't like it then the shows probably not for you.
- mindcrime 3y agoDamn, now I'm suddenly finding myself wanting to go back and re-watch Bojack Horseman. Not that that would be a bad thing.
- vslira 3y agoNot to be a skeptic or anything, but how do we know normal maps etc weren’t enriched into the datasets by the image gen companies? I understand this paper links to open source model where that can be verified, but maybe this is one secret sauce of these more advanced models?
- dougmwne 3y agoYou would need to train on pairings of normal map images to source images. To my knowledge, that’s not a common training technique and this ability seems to bridge across several open models.
- coffeebeqn 3y agoIt’s also kind of cool if it “understands” a normal map and that it maps to specific 3D geometry. People generally use 3D tools like Zbrush to sculpt the details to normals. Probably some people can draw them too
- auggierose 3y agoSo how do they get the normals, for example? Are they generated by the AI before the image is generated in order to generate the image, and they are just reading them off some internal state?
- corysama 3y agoThe training data is just billions of {RGB image, text description} pairs. So, it appears the model figured out how to make the normals as part of the process of making the images. Or, are you asking how the researchers extracted it? > I-LoRA modulates key feature maps to extract intrinsic scene properties such as normals, depth, albedo, and shading, using the models' existing decoders without additional layers, revealing their deep understanding of scene intrinsics.
- auggierose 3y agoYes, that sentence "modulates key feature maps" doesn't tell me anything. What do they mean when saying they extract the normals?
- dougmwne 3y agoYes, there’s rudimentary evidence that there’s essentially a 3D engine within the model that participates in generating the image. If we could inspect and interpret the whole process it would likely be bizarre and byzantine, like a sort of convergent evolution that independently recreates a tangled spaghetti mess of unreal engine, adobe lightroom, and a physical simulation of a Canon D5.
- qingcharles 3y agoEssentially similar perhaps to the 3D engine that a human brain runs that generates a single "3D" image from two 2D cameras (eyes) and fills in missing objects in blind spots, etc.
- tsimionescu 3y ago
- LarsDu88 3y agoReminds me of when I tried to extract G-buffers from my Unity High Definition Rendering Pipeline test project: https://www.youtube.com/watch?v=Fwtc694qNUM https://www.youtube.com/watch?v=Fwtc694qNUM I'm not sure if this paper is really proving anything though. There's a giant-ass UNET Lora model that's being trained here, so is it really "extracting" something from an existing model, or simply creating a new model that can create channels that look like something you'd get out of a deferred rendering pipeline. After all, taking normals, albedo, and depth and combining them (deferred rendering) is just one of several techniques to create a 3d scene. Wasn't even used in videogames until the early 2000s (in a Shrek videogame for the Xbox! (https://sites.google.com/site/richgel99/the-early-history-of-deferred-shading-and-lighting https://sites.google.com/site/richgel99/the-early-history-of...) What would really be awesome is to get a LORA model that can extract the "camera" rotation and translation matrix for these image generation models. That would really demonstrate something (and be quite useful at the same time).
- joefourier 3y agoIf you look at the supplementary material, they do a test where they train the Lora with a randomly initialised Unet and it is largely incapable of extracting any surface normals as opposed to using the pretrained Stable Diffusion Unet - clearly showing the features of the model are relevant to its performance.
- sfink 3y agoI don't really know what I'm talking about, but doesn't this address that? > with newly learned parameters that make up less than 0.6% of the total parameters in the generative model 0.6% sounds like a small number. Is it measuring the right thing? Certainly, I wouldn't expect the model to necessarily be encoding exactly the set of things that they're extracting, but it still seems very significant to me even if it is "just" encoding some set of things that can be cheaply (in terms of model size) and reliably mapped to normals, albedo, and depth. (I don't care what basis vectors it's using, as long as I know how to map them to mine.)
- LarsDu88 3y ago
- hn_throwaway_99 3y agoI skimmed the paper, but a lot of it was over my head. As someone not very versed in image generation AI, can anyone help me understand? This sentence (which another commenter highlighted) appears to be the key part: > I-LoRA modulates key feature maps to extract intrinsic scene properties such as normals, depth, albedo, and shading, using the models' existing decoders without additional layers, revealing their deep understanding of scene intrinsics. What exactly does "modulates key feature maps to extract intrinsic scene properties" mean? How were these scene property images generated if no additional decoding layers were added?
- visarga 3y agoSay you have a neural network with 1B parameters, you add 5M more parameters spread around (LoRA), and continue training for a while but only the newly added parameters, not the base network. The result is a "modulated" network that would predict scene properties. The interesting thing is that it only takes a few more parameters so it must be that the original network was pretty close already.
- kenjackson 3y agoGreat explanation! Wish I could upvote more as it took me from head scratching to “that makes total sense”.
- educaysean 3y agoThis is good news for VR (or spatial computing). If the models understand the physical world as well as the paper shows, generating two projections of a scene does not sound like a difficult ask. Really excited for what's to come.
- modeless 3y agoOoh, so this can take real images and predict albedo and lighting! Please, someone use this to make relightable gaussian splatting scenes. Dynamic lighting would really expand the usefulness of 3D scans made from photos, and I haven't seen anyone get anything close to what I would call "good" results in that area yet.
- IshKebab 3y agoCan it definitely use real images? I imagine extracting the depth map from real images would be the most useful application if it can.
- modeless 3y ago"Fortunately, diffusion models are not only powerful image generators, their structure as image-to-image models makes it straightforward to apply to real images." It looks like they actually only tried depth/normal maps for real images, not albedo/lighting maps, but it certainly seems possible. Honestly depth maps often look impressive at first glance but typically if you actually try to use them for anything (e.g. reprojection, DoF blur) their hidden flaws instantly become super apparent, and they aren't as useful as you might think. Gaussian splatting reconstructions already do reprojection better anyway. OTOH albedo maps look weird, but relighting is extremely useful and important. I'm not excited about yet another way to generate approximate depth maps, but I am excited about relighting.
- dvh 3y agoI asked ai that was posted here yesterday to draw common european chub and virtually every single one had adipose fin.
- LuciBb 3y ago[dead]
- nottorp 3y agoHmm. Facebook has had an option to create "3d photos" for a while. I think they're annoying but now I wonder how they're made, and if you really need a neural network to figure out the 3d part in a 2d photo.
- anjc 3y agoWould be interesting to see if the perceptive abilities of generative models are superior to human perception, when tested on optical illusions that humans are fooled by. E.g., do they correctly assess depth in a Ponzo illusion scenario
- renonce 3y agoSo is this GPT for images? They take a generative model and apply finetuning via LoRA on some downstream task such as surface normal and conclude that these models intrinsically learn these representations, and find that they do better than supervised approaches? I think this is awesome but maybe it’s not really that surprising given how this “generate and then finetune” approach has already worked so well?
- leobg 3y agoYet again. Anyone who placed their bets on LiDAR for self driving is proven wrong. Cameras and neural nets will rule the day.
- daxfohl 3y agoI wonder if you could do this on the NN-generating NN from https://news.ycombinator.com/item?id=39458363 https://news.ycombinator.com/item?id=39458363. Then you could have it start to create a vocabulary and an understanding around what all these parameters and weights do, since to us they're just a bunch of seemingly random floating point numbers. At which point we may be able to start creating NNs from first principles, without even requiring training.
- HellDunkel 3y agoDo we know the unet was never trained on gbuffers?