5 ms·
This comment is odd. I wouldn't say it is misleading, but it is odd because it borders on such definition. > Apple's AI strategy is to put inference (and longe
by SpaceManNabs 2y ago
This comment is odd. I wouldn't say it is misleading, but it is odd because it borders on such definition.
> Apple's AI strategy is to put inference (and longer term even learning) on edge devices
This is pretty much everyone's strategy. Model distillation is huge because of this. This goes in line with federated learning. This goes in line with model pruning too. And parameter efficient tuning and fine tuning and prompt learning etc.
> This is completely coherent with their privacy-first strategy
Apple's marketing for their current approach is privacy-first. They are not privacy first. If they were privacy first, you would not be able to use app tracking data on their first party ad platform. They shut it off for everyone else but themselves. Apple's approach is walled garden first.
> Processing data at the edge also makes for the best possible user experience because of the complete independence of network connectivity
as long as you don't depend on graph centric problems where keeping a local copy of that graph is prohibitive. Graph problems will become more common. Not sure if this is a problem for apple though. I am just commenting in general.
> If (and that's a big if) they keep their APIs open to run any kind of AI workload on their chips
Apple does not have a good track record of this; they are quite antagonistic when it comes to this topic. Gaming on apple was dead for nearly a decade (and pretty much still is) because steve jobs did not want people gaming on macs. Apple has eased up on this, but it very much seems that if they want you to use their devices (not yours) in a certain way, then they make it expensive to do anything else.
Tbf, I don't blame apple for any of this. It is their strategy. Whether it works or not, it doesn't matter. I just found this comment really odd since it almost seemed like evangelism.
edit: weird to praise apple for on device training when it is not publicly known if they have trained any substantial model even on cloud.
- nomel 2y ago> This is pretty much everyone's strategy. I think this is being too charitable on the state of "everyone". It's everyone's goal. Apple is actively achieving that goal, with their many year strategy of in house silicon/features.
- SpaceManNabs 2y ago> Apple is actively achieving that goal, with their many year strategy of in house silicon/features So are other companies, with their many year strategy of actually building models that accessible to the public. yet Apple is "actively" achieving the goal without any distinct models.
- deleted 2y ago[deleted]
- nomel 2y agoNo. "On edge" is not a model existence limitation, it is a hardware capability/existence limitation, by definition, and by the fact that, as you point out, the models already exist. You can already run those open weight models on Apple devices, on edge, with huge improvements on the newer hardware. Why is a distinct model required? Do the rumors appease these thoughts? If others are making models, with no way to actually run them, that's not a viable "on edge" strategy, since it involves waiting for someone else to actually accomplish the goal first (as is being done by Apple).
- SpaceManNabs 2y agoA week after this comment, google announced gemini nano locally in chrome.
- SpaceManNabs 2y ago> "On edge" is not a model existence limitation It absolutely is. Model distillation will still be pertinent. And so will be parameter efficient tuning for edge training. I cannot emphasize more how important this is. You will need your own set of weights. If apple wants to use open weights, then sure. Ignore this. Don't seem like they want to long-term... And even if they use open weights, they will still be behind other companies have done model distillation and federated learning for years. > Why is a distinct model required? Ask apple's newly poached AI hires this question. Doesn't seem like you would take an answer from me. > If others are making models, with no way to actually run them Is this the case? People have been running distilled llamas on rPis with pretty good throughput.
- jameshart 2y agoEveryone’s strategy? The biggest players in commercial AI models at the moment - OpenAI and Google - have made absolutely no noise about pushing inference to end user devices at all. Microsoft, Adobe, other players who are going big on embedding ML models into their products, are not pushing those models to the edge, they’re investing in cloud GPU. Where are you picking up that this is everyone’s strategy?
- SpaceManNabs 2y ago8 days after this comment, Google announced Gemini Nano local inference for Chrome.
- SpaceManNabs 2y ago> Where are you picking up that this is everyone’s strategy? Read what their engineers say in public. Unless I hallucinated years of federated learning. Also apple isn't even a player yet and everyone is discussing how they are moving stuff to the edge lol. Can't critique companies for not being on the edge yet when apple doesn't have anything out there.
- smj-edison 2y agoI believe at least Google is starting to do edge inference—take a look at the pixel 8 line-up they just announced. It doesn't seem to be emphasized as much, but the tensor G3 chip certainly has builtin inference.
- losvedir 2y agoOf course Google is. That's what Gemini Nano is for.