3 ms·
I agree, I've definitely seen way more information about running image synthesis models like Stable Diffusion locally than I have LLMs. It's counterintuitive to
by TheRealSteel 3y ago
I agree, I've definitely seen way more information about running image synthesis models like Stable Diffusion locally than I have LLMs. It's counterintuitive to me that Stable Diffusion takes less RAM than an LLM, especially considering it still needs the word vectors. Goes to show I know nothing.
I guess it comes down to the requirement of a very high end (or multiple) GPU that makes it impractical for most vs just running it in Colab or something.
Tho there are some efforts:
https://github.com/cocktailpeanut/dalai https://github.com/cocktailpeanut/dalai
- HexDecOctBin 3y agoEven the README of this project is lacking in details. They cover sizes of system RAM, but not of VRAM. Are they running the models on CPU exclusively? How does that compare to running them on GPUs? What are the corresponding numbers for GPUs? Not to mention more meta-topics, like which is preferred for inferencing - CPU or GPU? What are the corresponding numbers for fine-tuning or training for various model sizes? and so on.
- deleted 3y ago[deleted]
- james-revisoai 3y agoWell, if you looked at the AI world in 2019, the models were mostly less RAM intense (typically 0.8GB-2GB RAM for BERT/T5 non-large for example). And computer vision models always hit the max you'd throw at them. Things twist and turn. Text requires a large amount of parameters to be useful. If we as humans required 8000x8000 pictures to discern objects, images would require more. It's quite anthropomorphic. That's really the core of it.