4 ms·
Apple Intelligence Foundation Language Models
- TwentyPosts 2y agoWait, Apple releases papers? Do they have a name for their internal research division? When I think of companies doing AI research my mind generally jumps to Google DeepMind, I didn't know Apple releases any proper research papers at all.
- latexr 2y agohttps://machinelearning.apple.com/research https://machinelearning.apple.com/research
- rgovostes 2y agoFun fact: Their first paper, Improving the Realism of Synthetic Images (2017; https://machinelearning.apple.com/research/gan https://machinelearning.apple.com/research/gan), strongly hints at eye and hand tracking for the Apple Vision Pro released 5 years later.
- givinguflac 2y agoFor a while now! https://machinelearning.apple.com/research https://machinelearning.apple.com/research
- TwentyPosts 2y agoI believe it, I'm just surprised that I never thought of Apple as a big player here, and I'm wondering if this is an image problem on their side (rare, I know) or something else.
- dagmx 2y agoApple doesn’t like highlighting research or open source contributions in their marketing , likely because people speculate on what it means for products. There’s a ton of stuff out there from them that they intentionally appear to downplay because of that fear. Some of it is warranted though. Any little thing gets hundreds of speculative articles written and it’s hard to control a brand.
- deleted 2y ago[deleted]
- ein0p 2y agoAnd a number of them are actually quite good. Sometimes they even release the source for replication
- gmaster1440 2y ago[flagged]
- JumpCrisscross 2y ago> 1. Two Main Models: - AFM-on-device: ~3 billion parameters, for efficient on-device use - AFM-server: Larger model for Private Cloud Compute Could there be a new role for an Apple Server between these two? More capable than a phone but under my control.
- BoredPositron 2y agoSeeing how easy it is to have distributed loads between iPhones, iPads and Macs via MLX. It could really be the way forward for them.
- JumpCrisscross 2y ago> how easy it is to have distributed loads between iPhones, iPads and Macs via MLX Could you expand on this?
- BoredPositron 2y agoMLX is a Framework that lets you distribute ML workloads easily between nearly all modern Apple devices. We use it with some Mac Studios to get up to 1tb vram and for fun we sometimes throw in some iPhones :) https://ml-explore.github.io/mlx/build/html/index.html https://ml-explore.github.io/mlx/build/html/index.html
- ben_w 2y agoI may have missed something, but I only see the size of the smaller model, not the larger. Given how many parameters other models use, and how much Apple charges for RAM, my guess would be that a suitable server for the larger would be priced at around the same as my apartment. A mid-sized model… hm. In theory it could be anything, but in practice, this is Apple we're talking about. Probably still at least the price of a car.
- simonw 2y agoIf you were wondering what they trained it on: > The AFM pre-training dataset consists of a diverse dataset consists of a diverse and high quality data mixture. This includes data we have licensed from publishers, curated publicly-available or open-sourced datasets, and publicly available information crawled by our web-crawler, Applebot. We respect the right of webpages to opt out of being crawled by Applebot, using standard robots.txt directives. The fact that you can opt-out in robots.txt only if you knew to list Applebot months (years?) ago when they started crawling is a little unimpressive.
- seanmcdirmid 2y agoI thought robots.txt was provided an opt out for all web crawlers, not just vendor specific ones? What would be the use case for not using "*" if you didn't want something to be crawled?
- atty 2y agoIt’s very reasonable to want Google and Bing to index your page for search, but not have your data collected for training models, i think. I’m not familiar with robots.txt to know if it has a whitelisting mechanism
- seanmcdirmid 2y agoI see a lot of Robots.txt files that use non-wildcards for Allow but what would be the use case for using a non-widlcard for Disallow?
- simonw 2y agorobots.txt has always had mechanisms for allowing or denying specific crawlers. Most of the AI labs that crawl the web support something like this now, here are a few relevant examples from the NY Times robots.txt file for example: https://www.nytimes.com/robots.txt https://www.nytimes.com/robots.txt User-agent: anthropic-ai Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: FacebookBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: /
- deleted 2y ago[deleted]
- TradingPlaces 2y agoTrained on TPU4, not GPUs. The only GPUs involved are on Apple Silicon.