4 ms·
what hardware could one run this locally on? probably need 3090 etc right? finally a reason to upgrade from my m1 max
by tomr75 3y ago
what hardware could one run this locally on? probably need 3090 etc right?
finally a reason to upgrade from my m1 max
- redox99 3y ago34B is basically an exact fit for a 3090, when quantized to 4 bits.
- disconcision 3y agomaybe worth noting that depending on specifics a 3090 isn't necessarily a strict upgrade over an M1 max. While a 3090 will be faster on any model it can run, an M1 with 32GB of ram or more can load models which a 3090 cannot