3 ms·
The cost is coming down fast. You can get a $2000 desktop machine (AMD 395) that can run effectively chatGPT 3.5 levels of LLMs at over 100 tokens per second.
by fnord77 10mo ago
The cost is coming down fast. You can get a $2000 desktop machine (AMD 395) that can run effectively chatGPT 3.5 levels of LLMs at over 100 tokens per second.