3 ms·
We've not had cheap GPUs with this much VRAM before, though. Might be an interesting change, though I also doubt it personally.
by girvo 3mo ago
We've not had cheap GPUs with this much VRAM before, though. Might be an interesting change, though I also doubt it personally.
- nl 3mo agoAs noted in my other comment you can get obsolete GPUs (P100s, BC250s) with lots of RAM on AliExpress now. It hasn't proven revolutionary.
- swiftcoder 3mo agois 16Gb "lots of RAM" in an LLM world? How many of these would you need to stack on a motherboard to inference a decent size model?
- oblio 3mo agoAt least 4, probably 6+.
- girvo 3mo agoThose don't have a lot of RAM though. Not like these 40, 80GB ones.
- fc417fc802 3mo agoBC250 shares memory with system. 16 GB GPUs have long been available to consumers for a small premium. What's never been readily available before is 40+ GB of HBM.