2 ms·
I did not have great luck with 405b on an M1 Max Mac Studio with 64 GB of RAM via olama. I think memory was this issue. 70b runs with acceptable speed
by TradingPlaces 2y ago
I did not have great luck with 405b on an M1 Max Mac Studio with 64 GB of RAM via olama. I think memory was this issue. 70b runs with acceptable speed
- Lerc 2y ago64GB of ram is 550b bits of ram. That doesn't give you a lot of options on how many bits you want to keep per model parameter.