3 ms·
As noted in my other comment you can get obsolete GPUs (P100s, BC250s) with lots of RAM on AliExpress now. It hasn't proven revolutionary.
by nl 3mo ago
As noted in my other comment you can get obsolete GPUs (P100s, BC250s) with lots of RAM on AliExpress now. It hasn't proven revolutionary.
- swiftcoder 3mo agois 16Gb "lots of RAM" in an LLM world? How many of these would you need to stack on a motherboard to inference a decent size model?
- oblio 3mo agoAt least 4, probably 6+.
- girvo 3mo agoThose don't have a lot of RAM though. Not like these 40, 80GB ones.
- fc417fc802 3mo agoBC250 shares memory with system. 16 GB GPUs have long been available to consumers for a small premium. What's never been readily available before is 40+ GB of HBM.