6 ms·
I think your math is missing some details. An RTX 4090 has a memory bandwidth of 1,008 GB/s. PCIe 4 x16 has a 32GB/s bandwidth. DDR4 RAM has 3200 MT/s transf
by 0xbadc0de5 4y ago
I think your math is missing some details.
An RTX 4090 has a memory bandwidth of 1,008 GB/s.
PCIe 4 x16 has a 32GB/s bandwidth.
DDR4 RAM has 3200 MT/s transfer rate.
An AMD 5900x can easily max that out.
A good NVME can hit 7 GB/s read speeds or better.
And the 4-bit CUDA kernels can pack 16x 4-bit ints into a single 64-bit transfer / register.