Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
tmp777
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
3 ms
·
1.
▲
by
tmp777
3y ago
The big upside here is memory: you get 64G, which means you can easily run 70B models at 4bits. You'd need 3x4090s for that. And because of how most inference engines work today, the performance of such setup will actually be slightly
2.
▲
by
tmp777
3y ago
Keep in mind that you need at least the jetson board and the carrier board to make it work. The official nvidia devkit device with both boards is actually way more expensive that this (amazon lists it at $1,999). The jetson itself is pretty