yeah what, a GV100 is $1600 on eBay and you can run like a 3.8bpw quant of 70b llama at some decent number of tk/s, if you buy two you can run a 70b 5bpw quant

yeah what, a GV100 is $1600 on eBay and you can run like a 3.8bpw quant of 70b llama at some decent number of tk/s, if you buy two you can run a 70b 5bpw quant with 32k context no problem