4 ms·
The fundamental problem with this strategy is model size. I want all my apps to be privacy first with local models, but there is no way they can share models in
by strangescript 2y ago
The fundamental problem with this strategy is model size. I want all my apps to be privacy first with local models, but there is no way they can share models in any kind of coherent way. Especially when good apps are going to fine tune their models. Every app is going to be 3GB+
- tyho 2y agoFoundation models will be the new .so files.
- flawsofar 2y agoAnd fine tuning datasets will be compressed and sent rather than the whole model
- strangescript 2y agoThis would be interesting but also feels a little restrictive. Maybe something like LoRa could bridge the capability gap but if a competitor then drops a much more capable model then you either have to ignore it or bring it into your app. (assuming companies won't easily share all their models for this kind of effort)
- SpaceManNabs 2y agoI don't think HN understands how important model distillation still is for federated learning. Hype >> substance ITT
- Havoc 2y agoYou could always mix and match. Do lighter task on device and outsource to cloud if needed
- mr_toad 2y agoGemini nano is 1.8B 4 bit parameters, so a little under a GB. And hopefully each app won’t include a full copy of their models.
- int_19h 2y agoYou can do quite a lot with LoRA without having to replace all the weights.