3 ms·
nah, Lora quantized LLM’s are going to be at the OS level in 2 years and consumer architecture refreshes are just going to extend more RAM to already existing d
by yieldcrv 4y ago
nah, Lora quantized LLM’s are going to be at the OS level in 2 years and consumer architecture refreshes are just going to extend more RAM to already existing dedicated chips like Neural Engine
client side tokens per second will be through the roof and the models will be smaller
- computerex 4y agoLoRa is not a quantization method, it's a fine tuning method.
- yieldcrv 4y agoyou read that whole paragraph and assumed this prediction didn't involve consumers using fine tuned models despite lora being explicitly mentioned? -EQ moment edit: its about the combination of those methods making models accessible on consumer hardware
- robbintt 4y agoIt's two adjectives on the noun.