3 ms·
> Llama 3 70B, AMD 7800X3D: 1-2 Tokens/second How much RAM is required for this result? It's quite impressive that it even works as well as it does.
by zozbot234 2y ago
> Llama 3 70B, AMD 7800X3D: 1-2 Tokens/second
How much RAM is required for this result? It's quite impressive that it even works as well as it does.
- Netcob 2y agoI have 64 GB, but it really depends on the quantization. Looking at LM Studio I see versions ranging from 15 GB to 49 GB, and that's roughly how much RAM they will require. LM Studio will also let you do partial GPU offloads, but I've only started experimenting with that. The 1-2 Tokens/second value is what I got using GPT4All.