4 ms·
“We utilize adapters, small neural network modules that can be plugged into various layers of the pre-trained model, to fine-tune our models for specific tasks.
by scosman 2y ago
“We utilize adapters, small neural network modules that can be plugged into various layers of the pre-trained model, to fine-tune our models for specific tasks.”
This is huuuuge. I don’t see announcement of 3rd party training support yet, but I imagine/hope it’s planned.
One of the hard things about local+private ML is I don’t want every app I download to need GBs of weights, and don’t want a delay when I open a new app and all the memory swap happens. As an app developer I want the best model that runs on each HW model, not one lowest common denominator model for slowest HW I support. Apple has the chance to make this smooth: great models tuned to each chip, adapters for each use case, new use cases only have a few MB of weights (for a set of current base models), and base models can get better over time (new HW and improved models). Basically app thinning for models.
Even if the base models aren’t SOTA to start, the developer experience is great and they can iterate.
Server side is so much easier, but look forward to local+private taking over for a lot of use cases.
- dimtion 2y agoWith huge blobs of binary model weights, dynamic linking is cool again.
- inickt 2y agoWhich Apple has put some pretty large effort in the last few years to improve in iOS
- pjmlp 2y agoDynamic linking has always been cool for writing plugins. It is kind of ironic that languages that praise so much for going back to early linking models, have to resort for much heavier OS IPC for similar capabilities.
- rfoo 2y agoWhich languages? IIUC Go and Rust resort to OS IPC based plugin system mainly because they refused to have a stable ABI. On the other hand, at $DAYJOB we have a query engine written in C++ (which itself uses mostly static linking [1]) loading mostly static linked UDFs and ... it works. [1] Without glibc, but with libstdc++ / libgcc etc.
- pjmlp 2y agoWell if it loads code dynamically, it is no longer static linking. Also it isn't as if there is a stable ABI for C and C++ either, unless everything is compiled with the same compiler, or using Windows like dynamic libraries, or something like COM to work around the ABI limitations.
- skohan 2y agoDoesn’t rust’s static linking also have to do with the strategy of aggressive minimization? Iirc for instance every concrete instantiation of a dynamic type will have its own compiled binary, so it would basically be impossible for a dynamic library to do this since it wouldn’t know how it would be used, at least not without some major limitations or performance tradeoffs
- eightysixfour 2y agoThis is how Google is doing it too.
- gokuldas011011 2y agoIndeed. Google said LoRA and apple said adapter plugging. Wonder the difference is at, Apple's dev conference is for consumers and Google's dev conference is for developers.
- eightysixfour 2y agoYeah, just Apple doing a better job branding. LoRA is an adapter.
- scosman 2y agoOh missed that! But kinda as expected: only works on 2 android phones (pixel 8 pro, S24). Pretty typical: Apple isn’t first, but also typically will scale faster with HW+platform integration.
- rfoo 2y ago> will scale faster * Only in USA, both intentionally and not.
- Deathmax 2y agoOn Apple’s side, Apple Intelligence will only be enabled on A17 Pro and M-series chips, so only the iPhone 15 Pro and Pro Max will be supported in terms of phones.
- scosman 2y ago2 phones, ~4 tablets, ~12 PCs. Looking at sales, looks like about 10x the phone volume of s24 (and pixel 8 doesn’t register on the chats).
- danielmarkbruce 2y agothis is pretty stock standard lora.
- idiotsecant 2y ago[flagged]
- dang 2y ago"Don't be snarky." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- Sabinus 2y agoOne day they're going to train a moderation bot on your account, and it's going to be amazing.
- idiotsecant 2y agoYou caught me, I was weak in the face of ridiculousness. Keep on keeping on, you do a great job keeping this place halfway civil.
- danielmarkbruce 2y agoWhat was ridiculous? Are they not doing stock standard lora?
- lossolo 2y agoIt's LORA, most of the things you saw in Apple Intelligence on device presentation are basically different LORAs.
- scosman 2y agoThe article says it’s lora a bunch of times. That’s clear. My comment above is about dev experience, memory swapping, tuning base models to each HW release, and app size.
- seydor 2y agoLocal models are also extremely energy consuming. I don't see local AI working for long, because Large models are going to get so incomparably smarter and eventually reach general intelligence