3 ms·
if you're doing serious work, performance is more important than getting a good price/perf ratio, and a pair of 3090s is gonna be faster. It depends on your bud
by fragmede 2y ago
if you're doing serious work, performance is more important than getting a good price/perf ratio, and a pair of 3090s is gonna be faster. It depends on your budget,
however as that configuration is a bit more expensive, however.
- adam_arthur 2y agoWhether performance or cost is more important depends on your use case. Some tasks that an LLM can do very well may not need to be done often, or even particularly quickly (as in my case). e.g. LLM as one step of an ETL-style pipeline Latency of the response really only matters if that response is user facing and is being actively awaited by the user