3 ms·
The requirements say: NVIDIA 4090/5090 GPU 8GB+ VRAM (Full Version) I have a 3070 w 8GB of VRAM. Is there any reason I couldn’t run it (albeit slower) on my
by givinguflac 1y ago
The requirements say:
NVIDIA 4090/5090 GPU
8GB+ VRAM (Full Version)
I have a 3070 w 8GB of VRAM.
Is there any reason I couldn’t run it (albeit slower) on my card?
- GTP 1y agoJust try it and see.
- deleted 1y ago[deleted]
- mycall 1y agoWill this work with ROCm instead of CUDA?
- echelon 1y agoNo way. AMD is lightyears behind in software support.
- Zambyte 1y agoSpecifically for video? Ollama runs great on my 7900 XTX.
- deleted 1y ago[deleted]
- roenxi 1y agoThat isn't really what being behind implies. We've known how to multiply matrices since ... at least the 70s. And video processing isn't a wild new task for our friends at AMD. I'd expect that this would run on an AMD card. But I don't own an AMD card to check, because when I did it randomly crashed too often doing machine learning work.
- blkhawk 1y agoI have a 9070 XT... rockm ATM is unoptimized for it and the generation speed is less than what it should be if AMD isn't fudging the specs. Also the memory management is dire / buggy and will cause random OOMs on one ruin then be fine the next. splitting workflow helps so you can have one OOM crash in between. VAEs also crash from OOM. This is all just software issues because vram isn't released properly on AMD. *OOM = Out Of Memory Error
- snagadooker 1y ago2B model was running well on AMD, fingers crossed with 13B too: https://www.reddit.com/user/kejos92/comments/1hjkkmx/ltxv_inference_on_amd_gpus/ https://www.reddit.com/user/kejos92/comments/1hjkkmx/ltxv_in...
- zorgmonkey 1y agoSometimes it is a little more work to get stuff setup, but it works fine I've run plenty of models on my 7900 XTX wan2.1 14B, flux 1.dev and whisper. (wan and flux were with comfyui and whisper with whisper.cpp)
- turnsout 1y agoOr MLX/Apple?
- washadjeffmad 1y agoSure, just offload to system RAM, and don't use your system driver's fallback, but a specific implementation like MultiGPU. It won't speed it up, but using a quantization that fits in VRAM will prevent the offload penalty.