10 ms·
How Imagen Works
- coding123 4y agoIs this by a person that knows or is guessing?
- thunderbird120 4y agoGoogle published these implementation details
- tiborsaas 4y agoI guess he read the research paper.
- nestorD 4y agoThe paper is very well explained and, reading this post, they seems to mostly make its content accessible to non domain expert.
- watmough 4y agoThe important part seems to be the diffusion model. Explanation linked from same page: https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-introduction/ https://www.assemblyai.com/blog/diffusion-models-for-machine...
- DonHopkins 4y agoWait, this isn't about the line of intelligent xeroxographic laser printers developed by Imagen Corporation in 1981, supporting the Impress printer language? https://tug.org/TUGboat/tb02-2/tb03imagen.pdf https://tug.org/TUGboat/tb02-2/tb03imagen.pdf https://www.openprinting.org/driver/imagen https://www.openprinting.org/driver/imagen
- SleekEagle 4y agoHow do you think it prints the images!
- deleted 4y ago[deleted]
- dubswithus 4y agoIf Google has something similar or better it definitely makes it look like OpenAI is wasting its time. None of this relates to AGI.
- SleekEagle 4y agoI don't think anyone is saying that humanity is close to AGI, but check out DeepMind's Gato work for a more well-rounded agent: https://www.deepmind.com/publications/a-generalist-agent https://www.deepmind.com/publications/a-generalist-agent
- visarga 4y agoI think we're past a certain threshold, maybe not AGI but some definite qualitative change is happening.
- SleekEagle 4y agoI mean DALL-E 2 was the first time my jaw really hit the floor, although in fairness GPT-3 probably should've done that, but it's easier to do with images. And then for this to drop just a month later? Insane. It makes you wonder if they're actually releasing cutting edge, or Google decided to write this paper just because of the publication of DALL-E 2. Maybe they've had this model in the bag for a year.
- alphabetting 4y agoGoogle also released this different text to image model yesterday https://parti.research.google/ https://parti.research.google/ I think they've just got a lot of projects going on under the hood and timing was coincidence.
- SleekEagle 4y agoLooks cool although not as good as Imagen. Autoregressive vs Diffusion i guess
- natch 4y ago> Imagen, released just last month, can generate high-quality, high-resolution images given only a description of a scene “Released”? What? Papers are published. Websites are published. Tools are “released.” Where has Imagen been released?
- bpiche 4y agoThis implementation popped up on hacker news not too long ago. I got it working on Colab first, and then my own GPU at home. But just barely. Need more memory :) https://github.com/lucidrains/imagen-pytorch https://github.com/lucidrains/imagen-pytorch
- stavros 4y agoWait, so I can try this on Colab right now?
- refulgentis 4y agoNo, something that's been causing a lotta confusion in AI art is people stand up quick implementations generally matching the general description in the paper, but, they're not really investing in training them. Then people see "imagen-pytorch" on GitHub and get confused, either think it's Imagen itself or a suitable replica of it. There's like 3 projects named DallE, and then the 2 real DallEs...frustrating.
- Workaccount2 4y agoI have shown imagen (and dalle2) to a number of people now (non-tech, just everyday friends, family, co-workers) and I have been pretty stunned by the response I get from most people: "Meh, that's kinda cool? I guess?" or "What am I looking at?"..."Ok? So a computer made it? That seems neat" To me I am still trying to get my jaw off the floor from 2 months ago. But the responses have been so muted and shoulder shrugging that I think either I am missing something or they are missing something. Even really drilling in, practically shaking them "DO YOU NOT UNDERSTAND THAT THIS IS A ORIGINAL IMAGE CONSTRUCTED ENTIRELY BY AN AI?!?!" and people just seem to see it as a party trick at best.
- monkeybutton 4y agoPerhaps it's the combination of AI being so overhyped in the general public plus media that's already inundated with CGI, that it just doesn't blow them away?
- joshcryer 4y agoI've made perhaps overly absolutist statements like "don't you see! this kills artists jobs!" and it was shrugged off as if I was insane. I probably could've phrased it differently, but to me this is game changing in several fields. Granted, it will open up a new field of "generative artists" but, having played with these things, this is a pretty trivial job, and their training nets are only going to get better.
- Uehreka 4y agoI’ve had a lot of fun playing with Disco Diffusion prompts, but I agree that the people excited about “a generation of prompt artists” are a bit misguided. Soon an AI will emerge that can come up with “better” prompts than you, and the “art” of creating prompts will have a lower skill ceiling.
- deleted 4y ago[deleted]
- russdill 4y ago
- skinner_ 4y ago> The central intuition in using T5 is that extremely large language models, by virtue of their sheer size alone, may still learn useful representations despite the fact that they are not explicitly trained with any text/image task in mind. [...] Therefore, the central question being addressed by this choice is whether or not a massive language model trained on a massive dataset independent of the task of image generation is a worthwhile trade-off for a non-specialized text encoder. The Imagen authors bet on the side of the large language model, and it is a bet that seems to pay off well. The way out of this dilemma is to fine-tune T5 on the caption dataset instead of keeping it frozen. The paper notes that they don't do fine-tuning, but does not provide any ablation or other justification. I wonder if it would help or not.
- deleted 4y ago[deleted]
- varispeed 4y ago> is trained on hundreds of millions of images and their associated captions So how do you get access to hundreds of millions of images and use them to create derivative works? Did they get consent from millions of authors? Or is something like that only available to the rich with access to lawyers on tap? I mean I can imagine if a nobody wanted to do something like this, they'd get bankrupted by having to deal with all the photographers / artists spotting a tiny sliver of their art in the image produced by the model. Furthermore, would something like this work with music? For instance, train the model on all Spotify songs and then generate songs based on "Get me a Bach symphony played on sticks with someone rapping like Dr Dre with lisp." Or do music industry have enough money to bully anyone into not doing that?
- SleekEagle 4y agoPresumably Google's terms of service or fair use laws. The real restriction is that, even if you had the dataset, training costs tens of thousands of dollars. Only corporations can really afford to train these things. Regarding music - audio generation with Diffusion Models (the main component of Imagen and DALL-E 2) has been done, but not sure about music specifically. We will definitely reach the point where most e.g. pop beats will be able to be made by AI relatively soon. All a producer has to do is generate 100 beats and select the one s/he likes, potentially interpolate between 2 or finetune it.
- davikr 4y agoI've seen an image generated by AI contain an "Alamy" watermark before.
- astrange 4y agoThis is a real issue, but it's solvable with work. It's claimed that ML models' output isn't copyrightable because it's fair use, but that's hard to believe; a large model can easily memorise and output exactly one of its inputs again. This is easier to see with text, where GPT and Copilot both do it, but images can do it too. > So how do you get access to hundreds of millions of images and use them to create derivative works? Did they get consent from millions of authors? Build the model out of Creative Commons images only. There's a lot of 'em and it's good enough. You may need to exclude CC-BY since they currently can't follow the attribution requirement. > Or is something like that only available to the rich with access to lawyers on tap? More likely companies willing to license a stock photography database.
- alexccccc 4y agoSuper interesting
- funstuff007 4y agoWhat's the highest price paid for an AI-generated image NFT?
- lofatdairy 4y agoUnfortunately it seems like it's greater than 0... If we ignore the procedurally generated NFTs created from mixing and matching various assets and go with ones where AI is the selling point, we're left with a few notable ones: Sophia, a robot w/ some low-level AI sold a single piece for 689k USD [^1]. Botto, a VQGAN-based algorithm sold a single piece for 430k USD and has sold multiple other pieces for tens to hundreds of thousands of dollars. Slightly more modest are some other projects like Metascapes [^3] and Eponym [^4], which produced some really tedious pieces that managed to sell for 3.5k USD and 10k USD respectively. That said, the Eponym piece seems to be some sort of self promotion, so maybe we can say that the actual prices for these collections are somewhere in the fraction of an ETH range if they can be sold at all. Honestly, only the Botto piece is remotely interesting to look at, and even then I feel as if the blurred, "dreamy" aesthetic that seems to be in so many different AI painting approaches (style-transfer, VQGANS, DALL-E, maybe others I'm not aware of). I think it was more interesting back when we could pretend that these were the electric sheep at the fringes of some deep-sleeping latent intelligent potential but now they just feel kinda arbitrary and lacking deliberation. I absolutely love the field and think these researchers have done tremendous work, but I feel as though all the lay news attention is on the art, and not on the algorithm that generated it. The fascinating thing is that we have a machine that can produce novel something from words or basic ideas and that the output's content retains these ideas, not so much that art itself has that much compositional or stylistic merit. [^1]: https://niftygateway.com/itemdetail/primary/0xbe60d0a37ebde6f4ad22ceb311a28b3c53efe4e5/1 https://niftygateway.com/itemdetail/primary/0xbe60d0a37ebde6... [^2]: https://superrare.com/artwork-v2/scene-precede-29922 https://superrare.com/artwork-v2/scene-precede-29922 [^3]: https://opensea.io/assets/ethereum/0x75d639e5e52b4ea5426f2fb46959b9c3099b729a/1420 https://opensea.io/assets/ethereum/0x75d639e5e52b4ea5426f2fb... [^4]: https://opensea.io/assets/ethereum/0xaa20f900e24ca7ed897c44d92012158f436ef791/591 https://opensea.io/assets/ethereum/0xaa20f900e24ca7ed897c44d...
- astrange 4y agoIs there a compare and contrast between Imagen and Parti anywhere? I realize the paper came out yesterday, but maybe other people remember what "autoregressive" means better than I do.
- SleekEagle 4y agoUpon first inspection, Parti is not as good. This is perhaps unsurprising - in DALL-E 2 the prior model tested between autoregressive and diffusion models and the diffusion model outperformed
- aceon48 4y agoAI is now creative
- sagarpatil 4y agoI wonder how developers can monetise this? What use cases does it have?