2 ms·
> 3 billion parameters is a bit large for most smart devices, is it not? Not really. A 3B model quantized to 4 bits should run in any reasonable smartphone (us
by redox99 3y ago
> 3 billion parameters is a bit large for most smart devices, is it not?
Not really. A 3B model quantized to 4 bits should run in any reasonable smartphone (using around 2GB of memory).