4 ms·
Has anyone managed to run the thing? I got the streamlit demo to start after fighting with pytorch, mamba, and pip for half an hour, but the demo runs out of GP
by speedgoose 3y ago
Has anyone managed to run the thing? I got the streamlit demo to start after fighting with pytorch, mamba, and pip for half an hour, but the demo runs out of GPU memory after a little while. I have 24GB on GPU on the machine I used, does it need more?
- mkaic 3y agoHave heard from others attempting it that it needs 40GB, so basically an A100/A6000/H100 or other large card. Or an Apple Silicon Mac with a bunch of unified memory, I guess.
- mlboss 3y agoGive it a week.
- speedgoose 3y agoAlright thanks for the information. I will try to justify using one A100 for my "very important" research activities.
- skonteam 3y agoYeah, got a 24GB 4090, try to reduce the number of frames decoded to something like 4 or 8. Although, keep in mind it caps the 24Gb and goes to RAM (with the latest nvidia drivers).
- speedgoose 3y agoOh yes it works, thanks!
- nwoli 3y agoIs the checkpoint default fp16 or fp32?