3 ms·
There are Chinese modded 4090s with 48 and 96 GB of ram that seem like a sweet spot for fast inference of these moderately sized models.
by porphyra 2y ago
There are Chinese modded 4090s with 48 and 96 GB of ram that seem like a sweet spot for fast inference of these moderately sized models.
- BoorishBears 2y agoThe 96GB 4090 is still at the "blurry shots of nvidia-smi on Chinese TikTok" stage. Granted the 48 GB 4090s started there too before materializing and even becoming common enough to be on eBay, but this time there are more technical barriers and it's less likely they'll ever show up in meaningful numbers.
- noahbp 2y agoWe've progressed to the "more detailed photos and screen grabs on Twitter" stage: https://x.com/bdsqlsz/status/1898307273967145350 https://x.com/bdsqlsz/status/1898307273967145350
- johnisgood 2y agoFor how much? From where would one obtain them? Are they legit? I have found a lot of Chinese knock-offs for various GPUs for super cheap, but they were faulty (in my experience).
- qingcharles 2y agoIf you look on the local llama sub on Reddit there are threads about them; I bet there are links to the Chinese sellers.
- Tepix 2y agoThey are promising but they are priced very far beyond the 30€/GB of the used RTX 3090