4 ms·
Can anyone comment on the TinyBox they are taking preorders for? The tinybox 738 FP16 TFLOPS 144 GB GPU RAM 5.76 TB/s RAM bandwidth 30 GB/s model load band
by Mistletoe 3y ago
Can anyone comment on the TinyBox they are taking preorders for?
The tinybox
738 FP16 TFLOPS
144 GB GPU RAM
5.76 TB/s RAM bandwidth
30 GB/s model load bandwidth (big llama loads in around 4 seconds)
AMD EPYC CPU
1600W (one 120V outlet)
Runs 65B FP16 LLaMA out of the box (using tinygrad, subject to software development risks)
$15,000
- 1letterunixname 3y agoThere's a reason no one uses ATI GPUs in datacenters. Their dev support is shit. Don't waste your money. Buy 6 RTX 4090's and a decent ECC-memory server, and call it a day.
- wmf 3y agoDidn't read the article.
- guraf 3y agoI thought you weren't allowed to use Nvidia's consumer GPUs in the datacenter?
- evanchisholm 3y agoYou aren't, but who's going to stop you?
- interlinked 3y agoTheir closed source driver?
- ilyt 3y agoAnd how exactly they are going to know you're running card in DC ?
- wmf 3y agoWe can't really comment much on it because a bunch of specs are lacking. Is it using 6x 7900 XTXs? Which Epyc CPU (Epycs vary in price from $1K to $11K)?
- staunch 3y agoI like George's style and wish him well. But I'm not optimistic about their chances of selling $15k servers that are $10k in parts (or whatever the exact numbers are). It's just too easy for anyone to throw together a Supermicro machine with 6x GPUs in it, which is what it sounds like they'll be doing. My guess is they'll end up creating some premium extensions to the software and selling that to make money. Or maybe they can sell an enterprise cluster manager type thing that comes with support. He's good at software so it makes sense for him to sell software. And maybe the box will sell well initially just as a "dev kit" type thing.
- Animats 3y ago> And maybe the box will sell well initially just as a "dev kit" type thing. Price: $15,000. If they had a "lite" model that sold for $1500, and were actually shipping....
- wmf 3y agoThe lite model is any gaming PC with a 7900 XTX.
- wmf 3y agoTo me this looks like a way to mask donations.
- Shekelphile 3y ago> It's just too easy for anyone to throw together a Supermicro machine with 6x GPUs in it, which is what it sounds like they'll be doing. HPC compute is well advanced past just slapping GPUs into generic supermicro servers anyway. Without semi-custom hardware and equivalents to nvlink/nvswitch AMD won't ever be competitive in the HPC space.
- foobiekr 3y agoSuperMicro has HGX servers. $300k buys you an 8xH100 chassis with crazy amounts of memory, storage, CPU and GPU compute.
- quickthrower2 3y agoWhat is the cost of an equivalent setup using A100s? I have no idea what I am doing but here goes! By [1] we have 156 FP16 TFLOPS, taking their non "*" (* = with sparsity) value. So you need 5 So $40,000? pls the other stuff, and someone to make a profit putting it together say $50,000? So this setup is 3 times cheaper for the same. If I am allowed to use the sparsity value it is 1.5 times cheaper. [1] https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-us-nvidia-1758950-r4-web.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...