4 ms·
I'd love to know what kind of hardware would it take to do inference at the speed provided by the frontier model providers (assuming their models were available
by copperx 8mo ago
I'd love to know what kind of hardware would it take to do inference at the speed provided by the frontier model providers (assuming their models were available for local use).
10k worth of hardware? 50k? 100k?
Assuming a single user.