3 ms·
And local models are getting more accurate and efficient. And they really don't need a dedicated GPU for fast response, just really fast ram. New APUs such as
by derekp7 2y ago
And local models are getting more accurate and efficient. And they really don't need a dedicated GPU for fast response, just really fast ram. New APUs such as the AMD 395 are going to be game changers for local users with 256 GiB/s bandwidth.
- elorant 2y agoThe new Mac Studio is great for this job too. With the Max chip you get 128GBs of unified RAM for $3.5k. Memory bandwidth is at 546GB/s which is good enough for most tasks.