7 ms·
The 65B quantized model fits in 64GB of RAM, which I already had. Though RDIMMs on eBay are even cheaper than UDIMMs (just over $1/GB) and Broadwell-era Xeon w
by AnthonyMouse 4y ago
The 65B quantized model fits in 64GB of RAM, which I already had.
Though RDIMMs on eBay are even cheaper than UDIMMs (just over $1/GB) and Broadwell-era Xeon workstations aren't that expensive if you want to run the unquantized version.