3 ms·
What privacy benefit do you get running this locally vs renting a baremetal GPU and running it there? Wouldn't that be much more cost-effective? Especially wh
by onlyrealcuzzo 2y ago
What privacy benefit do you get running this locally vs renting a baremetal GPU and running it there?
Wouldn't that be much more cost-effective?
Especially when you inevitably want to run a better / different model in the near future that would benefit from different hardware?
You can get similar Tok/sec on a single RTX 4090 - which you can rent for <$1/hr.
- randomjoe2 2y agoBut at a totally different quant, you're crazy if you think you can run the entire R1 model on a single 4090, come on man. Apples and oranges.