4 ms·
I described this system word for word 2 years ago, glad to see it come to fruition on the only software stack integrated enough to do it. https://fleetwood.dev
by FL33TW00D 2y ago
I described this system word for word 2 years ago, glad to see it come to fruition on the only software stack integrated enough to do it.
https://fleetwood.dev/posts/a-case-for-client-side-machine-learning https://fleetwood.dev/posts/a-case-for-client-side-machine-l...
- jimmySixDOF 2y agoIf you described Speculative Decoding, 3bit Quantization, and Adapter based weight selection all two years ago then your AI Lab must be so sota /s
- reqo 2y agoI am curious, what do you mean by "Adapter based weight selection"?
- supriyo-biswas 2y agoThat's apparently Apple's term for LoRA[1]. [1] https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685
- jimmySixDOF 2y agoSort of but Adapters allow for multiple weight adjustments (think loras) for specific skills so it is more like extra optimized mixture of experts or multi agent approach. They have a slide with adapters listed like summarization, prioritization, tone (happy, business, etc), editor, etc) -- this is not to be mixed up with Intents which is how on device apps publish their capabilities to the Intelligence system for real npu os level multi agent tool use.
- aitooltime 2y agoThanks for posting the article. Some predictions in the article: 1. Partial client side execution of the models, then main model execution in cloud. (Example: text encoder for Stable Diffusion runs local, UNet runs cloud) 2. Local LoRAs for “dynamic ML” 3. WebGPU will be new standard to unlock models in the web Even though the article had some good calculations and overview, I am not sure word-for-word is how I would describe its relationship to the OP.
- T3RMINATED 2y ago[dead]