3 ms·
Local memory management will definitely get better in the future. For now: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B mode
by mchiang 3y ago
Local memory management will definitely get better in the future.
For now:
You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.
My personal recommendation is to get as much memory as you can if you want to work with local models [including VRAM if you are planning to be executing on GPU]
- bdavbdav 3y agoThanks - the issue I’m facing is the CTO lead times on Macs here!
- sourcecodeplz 3y agoI run the original llama 7b model just fine on 8GB of Ram. It is best to give advice from experience not only what you read from others.
- mchiang 3y agoThis is from us manually testing it on macbooks that we have available. It might run, but it's probably using swap.