4 ms·

Software developer that knows nothing about ML: What would it take to mess around with this? From a skim of the README it seems like install the package, and t

by djleni 4y ago

Software developer that knows nothing about ML: What would it take to mess around with this? From a skim of the README it seems like install the package, and then find some training dataset, and then ??? to use it.

mgalgs 4y ago
It looks like big-sleep [1] is their CLI? I did: pip install big-sleep dream "a pyramid made of basketballs" and after a few hours got this: https://i.imgur.com/FxdfdmV.png https://i.imgur.com/FxdfdmV.png Not nearly as cool as the real DALL-e, but maybe I'm missing something. [1] https://github.com/lucidrains/big-sleep https://github.com/lucidrains/big-sleep
moffkalast 4y ago
Well you most likely need a CUDA compatible GPU with a non-negligible amount of vram to load the model. And a few hours of your time to mash your head at the keyboard in frustration because of the inevitable dependencies that won't want to resolve properly. Most "open source" models don't actully upload the final models either, just the code they used to train them (not sure if this is the case here), so the next step after that is downloading 100G of data and running your workstation for a few days to train the actual model first. I recall at one point I attempted to run BERT or GPT-J or one of those lighter language models locally, and it was going well until I realized I needed a 24G of vRAM to even load the model hah.
- hadlock 4y ago
  Do CUDA AI projects not containerize well? I understand that AI projects often have many, many dependencies, but this seems like something containers ("docker") would be ideally suited for. Something like `docker pull dall-e` and `docker run -ie -n dalle dall-e` probably volume mount in a local folder? I'm curious what the hurdles are here for packaging something like this.
  - indiv0 4y ago
    They do containerize but it’s not enough. You still need the right drivers and CUDA stuff installed on the docker host, which can be finicky to setup. Not to mention figuring out how to get docker to actually pass the GPU to the container.
    - analyte123 4y ago
      I see this has not gotten any better in the last 5 years. For future reference, if you as a dev are interacting with a product that is this hard to use and there is no other option but to use it (CUDA), you should buy their stock.
      - danuker 4y ago
        I'd much rather look out for competitors. Resting on laurels won't last. If a company stops improving its offer, a competitor might catch up.
        ceeplusplus 4y ago
        The competition has even more broken software (AMD). At least Nvidia's works when you need it to.
        moffkalast 4y ago
        I think OpenCL support has been getting better, and it may be possible to run a lot of models with it, but that just doubles your already frustrating amount of hours trying to set the damn thing up.
    - boolemancer 4y ago
      > Not to mention figuring out how to get docker to actually pass the GPU to the container. Should just be passing `--gpus all` these days, shouldn't it?
  - sailingparrot 4y ago
- jaxrtech 4y ago
  Why not have some ML-rendition of BitTorrent designed for sharing pre-trained models? At least then you're not paying for AWS egress ha
  - jtvjan 4y ago
    That's not the issue. You can already download weights of other models through BitTorrent or from large archive sites like Hugging Face or the-eye. OpenAI just doesn't want to release them, so that they can sell access to it through their cloud service.
leereeves 4y ago
The readme says they're working on a command line interface (CLI). You might want to wait for that if you don't want to learn PyTorch. Once that's done it might be as simple as: install the package, and then find some training dataset, and then run the training CLI (for days or weeks or more), and then run the image generation CLI. Or even: just download an already trained model and use the image generation CLI immediately.
jorgemf 4y ago
This uses several models. One of them is already trained and released, which you can use or you can train it too yourself. The other models need training. Training means having a labelled dataset and let the model run several iterations until the loss function has a good value in order to use the model. This code is quite complex as it is not one model but several ones and also you can make some changes to generate bigger images and to use better models for some parts. Once you have everything trained (you might need a GPU or a cluster of GPUs to train it for several days/weeks/months), you can use it. I know this wont help you very much. But, either you are willing to spend a lot of time messing around with the code and have access to a good hardware to train the models, or you would have to wait until someone releases something already trained.
- visarga 4y ago
  > Once you have everything trained (you might need a GPU or a cluster of GPUs to train it for several days/weeks/months), you can use it. It takes a whole team to replicate an advanced model. Take a look at our open source working groups: Eleuther, LAION, BigScience. They worked for months for one release and burned millions of dollars on GPU (gracefully donated by well meaning sponsors). https://www.eleuther.ai/ https://www.eleuther.ai/ https://laion.ai/ https://laion.ai/ The BigScience group is massive: https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454bb8?v=92f51ef4650c4fc085cda88e23c8ebbb https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454...
  - shepardrtc 4y ago
    I didn't even know anything like those groups existed. That's very helpful. Thank you!
bogwog 4y ago
An older, but similar and still impressive alternative is available here: https://github.com/CompVis/latent-diffusion https://github.com/CompVis/latent-diffusion If you have a decent amount of VRAM, you can use it to start generating images with their pre-trained models. They're nowhere near as impressive as DALL-E 2, but they're still pretty damn cool. I don't know what the exact memory requirements are, but I've gotten it to run on a 1080 TI with 11gb. EDIT: I also tried a 980 with 4GB of RAM a while back, but that failed...so you probably need more than that.

by djleni 4y ago

mgalgs 4y ago

It looks like big-sleep [1] is their CLI? I did: pip install big-sleep dream "a pyramid made of basketballs" and after a few hours got this: https://i.imgur.com/FxdfdmV.png https://i.imgur.com/FxdfdmV.png Not nearly as cool as the real DALL-e, but maybe I'm missing something. [1] https://github.com/lucidrains/big-sleep https://github.com/lucidrains/big-sleep

moffkalast 4y ago

Well you most likely need a CUDA compatible GPU with a non-negligible amount of vram to load the model. And a few hours of your time to mash your head at the keyboard in frustration because of the inevitable dependencies that won't want to resolve properly. Most "open source" models don't actully upload the final models either, just the code they used to train them (not sure if this is the case here), so the next step after that is downloading 100G of data and running your workstation for a few days to train the actual model first. I recall at one point I attempted to run BERT or GPT-J or one of those lighter language models locally, and it was going well until I realized I needed a 24G of vRAM to even load the model hah.

hadlock 4y ago
Do CUDA AI projects not containerize well? I understand that AI projects often have many, many dependencies, but this seems like something containers ("docker") would be ideally suited for. Something like `docker pull dall-e` and `docker run -ie -n dalle dall-e` probably volume mount in a local folder? I'm curious what the hurdles are here for packaging something like this.
- indiv0 4y ago
  They do containerize but it’s not enough. You still need the right drivers and CUDA stuff installed on the docker host, which can be finicky to setup. Not to mention figuring out how to get docker to actually pass the GPU to the container.
  - analyte123 4y ago
    I see this has not gotten any better in the last 5 years. For future reference, if you as a dev are interacting with a product that is this hard to use and there is no other option but to use it (CUDA), you should buy their stock.
    - danuker 4y ago
      I'd much rather look out for competitors. Resting on laurels won't last. If a company stops improving its offer, a competitor might catch up.
      - ceeplusplus 4y ago
        The competition has even more broken software (AMD). At least Nvidia's works when you need it to.
        moffkalast 4y ago
        I think OpenCL support has been getting better, and it may be possible to run a lot of models with it, but that just doubles your already frustrating amount of hours trying to set the damn thing up.
  - boolemancer 4y ago
    > Not to mention figuring out how to get docker to actually pass the GPU to the container. Should just be passing `--gpus all` these days, shouldn't it?
- sailingparrot 4y ago
jaxrtech 4y ago
Why not have some ML-rendition of BitTorrent designed for sharing pre-trained models? At least then you're not paying for AWS egress ha
- jtvjan 4y ago
  That's not the issue. You can already download weights of other models through BitTorrent or from large archive sites like Hugging Face or the-eye. OpenAI just doesn't want to release them, so that they can sell access to it through their cloud service.

leereeves 4y ago

The readme says they're working on a command line interface (CLI). You might want to wait for that if you don't want to learn PyTorch. Once that's done it might be as simple as: install the package, and then find some training dataset, and then run the training CLI (for days or weeks or more), and then run the image generation CLI. Or even: just download an already trained model and use the image generation CLI immediately.

jorgemf 4y ago

This uses several models. One of them is already trained and released, which you can use or you can train it too yourself. The other models need training. Training means having a labelled dataset and let the model run several iterations until the loss function has a good value in order to use the model. This code is quite complex as it is not one model but several ones and also you can make some changes to generate bigger images and to use better models for some parts. Once you have everything trained (you might need a GPU or a cluster of GPUs to train it for several days/weeks/months), you can use it. I know this wont help you very much. But, either you are willing to spend a lot of time messing around with the code and have access to a good hardware to train the models, or you would have to wait until someone releases something already trained.

visarga 4y ago
> Once you have everything trained (you might need a GPU or a cluster of GPUs to train it for several days/weeks/months), you can use it. It takes a whole team to replicate an advanced model. Take a look at our open source working groups: Eleuther, LAION, BigScience. They worked for months for one release and burned millions of dollars on GPU (gracefully donated by well meaning sponsors). https://www.eleuther.ai/ https://www.eleuther.ai/ https://laion.ai/ https://laion.ai/ The BigScience group is massive: https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454bb8?v=92f51ef4650c4fc085cda88e23c8ebbb https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454...
- shepardrtc 4y ago
  I didn't even know anything like those groups existed. That's very helpful. Thank you!

bogwog 4y ago

An older, but similar and still impressive alternative is available here: https://github.com/CompVis/latent-diffusion https://github.com/CompVis/latent-diffusion If you have a decent amount of VRAM, you can use it to start generating images with their pre-trained models. They're nowhere near as impressive as DALL-E 2, but they're still pretty damn cool. I don't know what the exact memory requirements are, but I've gotten it to run on a 1080 TI with 11gb. EDIT: I also tried a 980 with 4GB of RAM a while back, but that failed...so you probably need more than that.