4 ms·

https://tinygrad.org/#tinybox https://tinygrad.org/#tinybox Looks like good value, but I wonder if it would get CPU/RAM bottlenecked, especially if you want to

https://tinygrad.org/#tinybox https://tinygrad.org/#tinybox Looks like good value, but I wonder if it would get CPU/RAM bottlenecked, especially if you want to train something with a lot of preprocessing in the pipeline. Something comparable I've found with 7x4090 which comes to about $50k, but with much better CPU/RAM (3x CPU, 4x RAM, 5x SSD): https://www.overclockers.co.uk/8pack-supernova-mk3-amd-ryzen-threadripper-pro-extreme-pc-sys-8pk-00076.html https://www.overclockers.co.uk/8pack-supernova-mk3-amd-ryzen...

gpjt 2y ago
Closer to $42k, i think, at least if you're comparing it to the Tinybox price -- the price in pounds on the site includes VAT, which you wouldn't pay as a business or if you were getting it for export outside the UK, whereas you'd need to add on VAT if you were getting a Tinybox in the UK.
SushiHippie 2y ago
> 6x PCIe 4.0 x16 (64 GB/s) Wikipedia [0] states that PCIe 4.0 x16 has a throughput of ~32GB/s, what does the (64 GB/s) indicate on the website, is this just a typo and you have 6x ~32GB/s or does it mean in total you can "only" expect a throughput of 64GB/s all lanes slots combined? If so, wouldn't you also be bottlenecked by the PCIe bandwidth (when moving data between CPU and GPU)? [0] https://en.wikipedia.org/wiki/PCI_Express#Comparison_table https://en.wikipedia.org/wiki/PCI_Express#Comparison_table
- arder 2y ago
  Well they're specifying the AMD EPYC and one of the things that the server line of AMD CPUs do that the consumer grade ones don't, is they have lots of connectivity. So for example an AMD EPYC 8324P is a 32 core CPU with 96 lanes of PCI Gen 5. Given that the 4090 GPU is PCI Gen 4, I think that's where you get the discrepancy. The 6 GPUS are connected in parallel to the CPU with 6 x16 connections (96 total lanes), the CPU could do this at Gen 5 (64GBs for each GPU) but the 4090 GPU is Gen4 only, so you'll only actually get 32GBps per connection.
- jasomill 2y ago
  Most EPYCs have 128 PCIe lanes, so I'd expect a full x16 link for all six GPUs. Pedantically, the combined bidirectional bandwidth of PCIe x16 is ~64 GB/s, as it's a full-duplex ~32 GB/s link, but that's an awfully misleading spec if this is the intent (akin to claiming Gigabit Ethernet is 2 Gb/sec).
  - georgehotz 2y ago
    It's the same way NVIDIA states bandwidth for PCIe and NVLink. https://www.nvidia.com/en-us/data-center/h100/ https://www.nvidia.com/en-us/data-center/h100/
- andersa 2y ago
  It's 32GB/s in both directions. So when exchanging data two GPUs each can do this at 64GB/s. Is that a useful way to measure it? Who knows.
coder543 2y ago
It’s weird how non-specific the CPU is there. Why wouldn’t they list a CPU part number? We don’t even know what generation of Epyc it is. (I get that it’s not the focus… but it is still important.)

gpjt 2y ago

Closer to $42k, i think, at least if you're comparing it to the Tinybox price -- the price in pounds on the site includes VAT, which you wouldn't pay as a business or if you were getting it for export outside the UK, whereas you'd need to add on VAT if you were getting a Tinybox in the UK.

SushiHippie 2y ago

> 6x PCIe 4.0 x16 (64 GB/s) Wikipedia [0] states that PCIe 4.0 x16 has a throughput of ~32GB/s, what does the (64 GB/s) indicate on the website, is this just a typo and you have 6x ~32GB/s or does it mean in total you can "only" expect a throughput of 64GB/s all lanes slots combined? If so, wouldn't you also be bottlenecked by the PCIe bandwidth (when moving data between CPU and GPU)? [0] https://en.wikipedia.org/wiki/PCI_Express#Comparison_table https://en.wikipedia.org/wiki/PCI_Express#Comparison_table

arder 2y ago
Well they're specifying the AMD EPYC and one of the things that the server line of AMD CPUs do that the consumer grade ones don't, is they have lots of connectivity. So for example an AMD EPYC 8324P is a 32 core CPU with 96 lanes of PCI Gen 5. Given that the 4090 GPU is PCI Gen 4, I think that's where you get the discrepancy. The 6 GPUS are connected in parallel to the CPU with 6 x16 connections (96 total lanes), the CPU could do this at Gen 5 (64GBs for each GPU) but the 4090 GPU is Gen4 only, so you'll only actually get 32GBps per connection.
jasomill 2y ago
Most EPYCs have 128 PCIe lanes, so I'd expect a full x16 link for all six GPUs. Pedantically, the combined bidirectional bandwidth of PCIe x16 is ~64 GB/s, as it's a full-duplex ~32 GB/s link, but that's an awfully misleading spec if this is the intent (akin to claiming Gigabit Ethernet is 2 Gb/sec).
- georgehotz 2y ago
  It's the same way NVIDIA states bandwidth for PCIe and NVLink. https://www.nvidia.com/en-us/data-center/h100/ https://www.nvidia.com/en-us/data-center/h100/
andersa 2y ago
It's 32GB/s in both directions. So when exchanging data two GPUs each can do this at 64GB/s. Is that a useful way to measure it? Who knows.

coder543 2y ago

It’s weird how non-specific the CPU is there. Why wouldn’t they list a CPU part number? We don’t even know what generation of Epyc it is. (I get that it’s not the focus… but it is still important.)