2 ms·
Yes if by "smart device" you mean iot or watch. A quantized 7B can fit on a high end iphone (and perform well) though so all relative
by Havoc 3y ago
Yes if by "smart device" you mean iot or watch. A quantized 7B can fit on a high end iphone (and perform well) though so all relative
- MacsHeadroom 3y agoI run 14b models locally on my Android phone. 3b would be much much faster though. 3b is also small enough to fit in a wasm runtime for browser based text local text generation.