5 ms·

A few comments by someone who's spent way too much time in the AI-generated space: * I recommend reading the Risks and Limitations section that came with it be

A few comments by someone who's spent way too much time in the AI-generated space: * I recommend reading the Risks and Limitations section that came with it because it's very through: https://github.com/openai/dalle-2-preview/blob/main/system-card.md https://github.com/openai/dalle-2-preview/blob/main/system-c... * Unlike GPT-3, my read of this announcement is that OpenAI does not intend to commercialize it, and that access to the waitlist is indeed more for testing its limits (and as noted, commercializing it would make it much more likely lead to interesting legal precedent). Per the docs, access is very explicitly limited: (https://github.com/openai/dalle-2-preview/blob/main/system-card.md#access https://github.com/openai/dalle-2-preview/blob/main/system-c... ) * A few months ago, OpenAI released GLIDE ( https://github.com/openai/glide-text2im https://github.com/openai/glide-text2im ) which uses a similar approach to AI image generation, but suspiciously never received a fun blog post like this one. The reason for that in retrospect may be "because we made it obsolete." * The images in the announcement are still cherry-picked, which is therefore a good reason why they tested DALL-E 1 vs. DALL-E 2 presumably on non-cherrypicked images. * Cherry-picking is relevant because AI image generation is still slow unless you do real shenanigans that likely compromise image quality, although OpenAI has likely a better infra to handle large models as they have demonstrated with GPT-3. * It appears DALL-E 2 has a fun endpoint that links back to the site for examples with attribution: https://labs.openai.com/s/Zq9SB6vyUid9FGcoJ8slucTu https://labs.openai.com/s/Zq9SB6vyUid9FGcoJ8slucTu

YeGoblynQueenne 5y ago
Regarding cherry-picking, the images of astronauts on horses look stunning, except for their hands. There's something seriously wrong with their hands. Maybe give it another five years, a few more $billion and a few more petabytes/flops and it will be good. Then finally everyone can generate art for their own Magic: the Gathering cards. (That's the end goal, right?)
- a_bonobo 5y ago
  Interestingly, hands are also something humans struggle to draw. They're a very complex anatomical form, many small tendons and muscles. Many artists struggle to depict hands. They're not made out of a few straight lines like a torso, there's lots of skew going on. They're probably the hardest structure of the human body to 'learn' for a ML system.
  - matthew-wegner 5y ago
    I think some of this is because hands are very involved in both communication and threat assessment, so we as humans put a lot of automatic attention on them. We aren't even usually aware of it--unless something looks off Personally I really like leaning into the discontinuities and quirkiness of generated images. This is output from GLID-3: https://twitter.com/mwegner/status/1511139661095178241 https://twitter.com/mwegner/status/1511139661095178241
- gwern 5y ago
  As I keep telling people: "hands are hard". This is why I went so far as to make a hand-specific dataset ("PALM" https://www.gwern.net/Crops#palm https://www.gwern.net/Crops#palm which of course now everyone is going to confuse with 'PaLM'...). Hands are just way too variable to learn easily. My dataset is a start, but it may benefit from focused training, the way Facebook's new Make-A-Scene https://arxiv.org/abs/2203.13131#facebook https://arxiv.org/abs/2203.13131#facebook (not DALL-E 2 quality but not far from it) has focused losses on faces.
- yokto 5y ago
  Hands are notoriously hard to even photograph. You very quickly get weird unnatural results with a camera in front of hands, so in a way I'm not surprised AI models struggle to produce satisfying imagery there too.
- MichaelDickens 5y ago
  Just a thought, I wonder if DALL-E would generate more realistic hands if you prompted it with something like > an astronaut riding a horse, and the astronaut has five fingers on each hand
CivBase 5y ago
The Risks and Limitations section is particularly interesting to me. It's like a time capsule of society's current fears about technology. They talk about many ways this tech could be misused, but I don't think they've even scratched the surface. An example off the top of my head: this could be used as advertising or recruitment for controversial organizations or causes. Would it be wrong for the USA to use this for military recruitment? Israel? Ukraine? Russia? Another example: this could be used to glorify and reinforce actions which our society does not consider to immoral but other societies - or our own future society - will. It wasn't long ago that the US and Europe did a full 180 on their treatment of homosexuality. Will we eventually change our minds about eating meat, driving cars, etc? Have they gone too far in a desperate bid to prevent the AI from being capable of harm? Have they not gone far enough? I don't know. If I was that worried about something being misused, I don't think I could ever bring myself to work on it in the first place. But I suppose the onward march of technology is inevitable.
bufferoverflow 5y ago
Not-so-open.ai
- qeternity 5y ago
  open-your-wallet.ai
- btdmaster 5y ago
  https://www.eleuther.ai https://www.eleuther.ai (text, not images, but free as in freedom)
  - deleted 5y ago
    [deleted]
  - refulgentis 5y ago
    Katherine Crawson is @ Eletheur & IMHO is indisputably most responsible for the advances in text=>image generation. Dall-E 2 is Dall-E and her insight to use diffusion, the intermediate proof of concept of diffusion + Dall-E is GLIDE. https://twitter.com/RiversHaveWings https://twitter.com/RiversHaveWings & https://github.com/crowsonkb https://github.com/crowsonkb
    - learndeeply 5y ago
      Diffusion models existed long before this announcement. I have no idea who this person is, but they did not invent this idea. Edit: Diffusion models guided by CLIP*
      - refulgentis 5y ago
        They did invent the idea of applying it to image generation, leading to OpenAI citing her _tweets_ (how cool is that?) in a paper for GLIDE, which as other comments note, looks just like a proof of concept of DallE-2.
    - djoldman 5y ago
      The paper cites K. Crowson 4 times: [5] Katherine Crowson. Ava linear probe. https://twitter.com/RiversHaveWings/status/ https://twitter.com/RiversHaveWings/status/ 1472346186728173568?s=20&t=T-HRr3Gw5HRGjQaMDtRe3A, 2021. [6] Katherine Crowson. Clip guided diffusion hq 256x256. https://colab.research.google.com/ https://colab.research.google.com/ drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj, 2021. [7] Katherine Crowson. Clip guided diffusion 512x512, secondary model method. https://twitter https://twitter. com/RiversHaveWings/status/1462859669454536711, 2021. [8] Katherine Crowson. v-diffusion. https://github.com/crowsonkb/v-diffusion-pytorch https://github.com/crowsonkb/v-diffusion-pytorch, 2021.

YeGoblynQueenne 5y ago

Regarding cherry-picking, the images of astronauts on horses look stunning, except for their hands. There's something seriously wrong with their hands. Maybe give it another five years, a few more $billion and a few more petabytes/flops and it will be good. Then finally everyone can generate art for their own Magic: the Gathering cards. (That's the end goal, right?)

a_bonobo 5y ago
Interestingly, hands are also something humans struggle to draw. They're a very complex anatomical form, many small tendons and muscles. Many artists struggle to depict hands. They're not made out of a few straight lines like a torso, there's lots of skew going on. They're probably the hardest structure of the human body to 'learn' for a ML system.
- matthew-wegner 5y ago
  I think some of this is because hands are very involved in both communication and threat assessment, so we as humans put a lot of automatic attention on them. We aren't even usually aware of it--unless something looks off Personally I really like leaning into the discontinuities and quirkiness of generated images. This is output from GLID-3: https://twitter.com/mwegner/status/1511139661095178241 https://twitter.com/mwegner/status/1511139661095178241
gwern 5y ago
As I keep telling people: "hands are hard". This is why I went so far as to make a hand-specific dataset ("PALM" https://www.gwern.net/Crops#palm https://www.gwern.net/Crops#palm which of course now everyone is going to confuse with 'PaLM'...). Hands are just way too variable to learn easily. My dataset is a start, but it may benefit from focused training, the way Facebook's new Make-A-Scene https://arxiv.org/abs/2203.13131#facebook https://arxiv.org/abs/2203.13131#facebook (not DALL-E 2 quality but not far from it) has focused losses on faces.
yokto 5y ago
Hands are notoriously hard to even photograph. You very quickly get weird unnatural results with a camera in front of hands, so in a way I'm not surprised AI models struggle to produce satisfying imagery there too.
MichaelDickens 5y ago
Just a thought, I wonder if DALL-E would generate more realistic hands if you prompted it with something like > an astronaut riding a horse, and the astronaut has five fingers on each hand

CivBase 5y ago

The Risks and Limitations section is particularly interesting to me. It's like a time capsule of society's current fears about technology. They talk about many ways this tech could be misused, but I don't think they've even scratched the surface. An example off the top of my head: this could be used as advertising or recruitment for controversial organizations or causes. Would it be wrong for the USA to use this for military recruitment? Israel? Ukraine? Russia? Another example: this could be used to glorify and reinforce actions which our society does not consider to immoral but other societies - or our own future society - will. It wasn't long ago that the US and Europe did a full 180 on their treatment of homosexuality. Will we eventually change our minds about eating meat, driving cars, etc? Have they gone too far in a desperate bid to prevent the AI from being capable of harm? Have they not gone far enough? I don't know. If I was that worried about something being misused, I don't think I could ever bring myself to work on it in the first place. But I suppose the onward march of technology is inevitable.

bufferoverflow 5y ago

Not-so-open.ai

qeternity 5y ago
open-your-wallet.ai
btdmaster 5y ago
https://www.eleuther.ai https://www.eleuther.ai (text, not images, but free as in freedom)
- deleted 5y ago
  [deleted]
- refulgentis 5y ago
  Katherine Crawson is @ Eletheur & IMHO is indisputably most responsible for the advances in text=>image generation. Dall-E 2 is Dall-E and her insight to use diffusion, the intermediate proof of concept of diffusion + Dall-E is GLIDE. https://twitter.com/RiversHaveWings https://twitter.com/RiversHaveWings & https://github.com/crowsonkb https://github.com/crowsonkb
  - learndeeply 5y ago
    Diffusion models existed long before this announcement. I have no idea who this person is, but they did not invent this idea. Edit: Diffusion models guided by CLIP*
    - refulgentis 5y ago
      They did invent the idea of applying it to image generation, leading to OpenAI citing her _tweets_ (how cool is that?) in a paper for GLIDE, which as other comments note, looks just like a proof of concept of DallE-2.
  - djoldman 5y ago
    The paper cites K. Crowson 4 times: [5] Katherine Crowson. Ava linear probe. https://twitter.com/RiversHaveWings/status/ https://twitter.com/RiversHaveWings/status/ 1472346186728173568?s=20&t=T-HRr3Gw5HRGjQaMDtRe3A, 2021. [6] Katherine Crowson. Clip guided diffusion hq 256x256. https://colab.research.google.com/ https://colab.research.google.com/ drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj, 2021. [7] Katherine Crowson. Clip guided diffusion 512x512, secondary model method. https://twitter https://twitter. com/RiversHaveWings/status/1462859669454536711, 2021. [8] Katherine Crowson. v-diffusion. https://github.com/crowsonkb/v-diffusion-pytorch https://github.com/crowsonkb/v-diffusion-pytorch, 2021.