Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
paulzain
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
4 ms
·
1.
▲
by
paulzain
3y ago
I run CodeLlama70B in my home lab using two RTX 4090s and the MLC AI framework. It provides Muti-GPU support and is extremely fast on consumer grade hardware. I'm seeing about 30 tokens/sec. https://blog.mlc.ai/202