3 ms·
I have zero qualifications to talk about this subject, but from what little I do know it's because Stable Diffusion's network has "only" ~900m parameters, where
by DaiPlusPlus 4y ago
I have zero qualifications to talk about this subject, but from what little I do know it's because Stable Diffusion's network has "only" ~900m parameters, whereas ChatGPT has 175 billion parameters.
- flangola7 4y agoI wonder what a 100B image model will create.
- cyanf 4y agoImages, most likely.
- tomohelix 4y agoI dread that. You heard about Loab? That stuff is haunting. I am scared of what a 100B param model can create in the deep recesses of its system.
- DaiPlusPlus 4y agoI thought Loab was just a creeopypasta?
- JeremyBanks 4y agoNo, Loab was an extreme value in parameter space which certain inputs could converge too.
- p-e-w 4y agoSure, but that's just another way of saying SD needs less space than ChatGPT, right? The question is why an image-generating model needs so much fewer parameters than a text-generating model in order to produce useful results, when our everyday experience teaches us that images need much more storage space than text to convey similar information.
- zamnos 4y agoI think that's because the mind can make more sense of a mismash of visual data and see patterns and pictures. Comparatively if drop words here or you can still make sense of what I wrote, but it's much more difficult for the reader to parse what I'm saying if I accidentally a few words. If my drawing has a clearly recognizable human figure but it has 7 fingers, it's still recognizable that it was trying to draw a human. LLMs have been able to generate words for years now. Hell, Tay was back in 2016. Making sure ChatGPT is able to answer the way it does (ie filter out bad things) is part of what makes it hard, and thus bigger, to implement. But having a cohesive readable output is what's hard and takes up more space. Plus, unless I missed it, we don't actually know how many gigabytes the model file for ChatGPT is.
- tomohelix 4y ago>images need much more storage space than text to convey similar information That is because an image is a collection of bit that attempt to represent reality as it is. It probably is easier to relates blue as a "color archetype" when you literally have a collection of bits that mean literal "blue" all the time. In languages "blue" doesn't always mean the color. Texts are abstractions/coded form of reality. Your mind itself contains the decryption codes to translate text into what it actually means. That decryption code for text apparently is much bigger and harder for a machine to crack than interpreting images. Also, images are bigger than a text file because of the way data is stored. It probaby has nothing to do with the amount of information stored inside. A book about quantum physics can be smaller than an image of a cat for example.
- famouswaffles 4y agoThe mistake you're making is thinking of weights as an avenue to store training data. This is wrong. I've hopefully explained it more in another comment but weights/neurons are essentially configuration settings.
- swatcoder 4y agoIt could suggest that humans have a much more sensitive and expressive sense of language than they do image perception, which seems plausible. We can spot flawed language in more contexts than flawed images, and can produce a bigger range of representation in text and than image. So for a system to produce human-satisfying text, it needs to be far more prepared than one that produces images. Conversely, this may underscore how inefficient pixel-like storage is for communicative and artistic images. Ten years from now, many of those kinds of images may only take a few hundred bytes and a good enough generator model to “decompress” them for display.