15 ms·

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 l

• bottlepalm 4y ago
  Maybe we should launch 100 of them out into space in different directions. Very low mass means we should be able to push it to a pretty high velocity.
  • LASR 4y ago
    > into space in different directions Different directions - this is the fundamental challenge. It's hard to know what these directions are when we're flying blind at the cutting edge of technology. We're all human. I suspect out of the 100, you'll have 95 of them going into the same as before areas. Maybe 5 truly understand the above point and explicitly strike down ideas that have been done before. GPT-3 has been live for a while now. The industry has hundreds of story generator apps specialized for various kinds of content generation. Very few are really thinking about AGI or chain-of-thought reasoning etc as an example. So I don't think launching 100 of them into different directions would work. Maybe 5 of them with both the expertise and direction to pursue seemingly impossible goals.
  • simonh 4y ago
    I know some people don’t like iPhones but this is taking it a bit far.
• ll_mama 4y ago
  5 years? Bit of a long stretch with how much focus is going into AI. I’d be surprised if it’s not possible within two iterations of the new iPhone.
  • Gigachad 4y ago
    GPT-3 was said to require something like 150gb of VRAM. I don't see that gap being bridged in phones within 2 years.
    • mlboss 4y ago
      LLaMA already runs on M2 and is comparable to gpt 3
      • crooked-v 4y ago
        I gave a try at Alpaca-LoRA with tloen's tuning and it definitely feels in reach of GPT 3. Not quite as good, but some of that may just be in whatever's going on with OpenAI's hidden prompts encouraging lots of text out of the model.
    • PaulDavisThe1st 4y ago
      Not impossible that the gap will be bridged in the other direction, by GPT-N or a cousin requiring much less VRAM ....
    • pbronez 4y ago
      Alpaca already works in just 4GB of RAM. This stuff is moving incredibly fast.
      • astrange 4y ago
        It doesn't work well at all. (Also alpaca.cpp uses 8GB of RAM for a 4GB model, possibly because they don't know how mmap works.)
        • dachryn 4y ago
          alpaca lora is quite capable actually and is suddenly gaining a hell of a lot of traction in the corporate world
    • dragonwriter 4y ago
• davnicwil 4y ago
  I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. We must be talking orders of magnitude differences in operational cost, not to mention completely unique features like privacy. The very definition of disruption, waiting in the wings. Not only that, but this model of 'wait in the wings and pounce when the tech is really there and absolutely nail it' is Apple's forté, aligned completely with the cultural and strategic DNA of the company.
  • jorvi 4y ago
    Siri was supposed to become locally processed but sometimes can’t even set a timer because of a connection time-out. Or she’ll use her fancy ML speech recognition model to turn “set a timer for 3 minutes 10 seconds” into “search for trinity tensor”. So much for an “unfathomable moat”.
    • getcrunk 4y ago
      You see the only reason it does that is because your … holding your phone wrong
    • belugacat 4y ago
      Siri is so bad it needs to be scrapped and rethought from the ground up (it can’t even give me the time when internet is down at my rural house). No one at Apple who wants a career there would dare propose that. The switch to a LLM architecture could be the perfect transition point for this.
      • 015a 4y ago
        If Siri can't give you the time without the internet, I think you need to update iOS. There are definitely two tiers of Siri queries. There are queries like "set the brightness to 10%" or "set a timer for 5 minutes" which absolutely and consistently work without internet, and have for several years, and if you're legitimately having a different experience then its possible a cosmic ray hit your iPhone (or, realistically, you're running into a strange and rare bug which is not indicative of the general experience and will be fixed in three to five years or maybe never). There are also queries like "create a new note" (from the Notes app) which should be able to work offline, but don't. And, naturally, there are queries like "when did Resident Evil 4 come out" which wouldn't reasonably work without internet (but, if you're curious, she does get it right). In other words, Siri clearly does some local guessing as to whether she can answer a query without the internet, and some queries appear to be miscategorized into the second bucket. My leading theory on why this happens, which may be incorrect, but it seems like: if an app has any Siri functionality which requires the internet to answer, all of the queries which are responded to by that app have to require the internet. It doesn't matter where the processing ends up actually happening to respond to the query, it just shuts the query down. Its weird, but its consistent with the behavior I've seen. The more important point: Siri's real and weird limitations don't seem to have much to do with limitations in local processing. They've said that the speech interpretation all happens on-device. They encrypt practically all of your data that does get shipped to their servers, so a query like "Open the note titled 'Hello World'" is probably also being processed on-device. But: Siri still requires the internet for that query. That doesn't seem like a significant limitation with their ML algorithms or silicon or anything meaningful; it seems like just a case of dumb coding, which is certainly something Apple is no stranger to.
• hokkos 4y ago
  But users will need to store multiples TB of data to run it ?
• smoldesu 4y ago
  I mean, it's possible to load LLMs onto a smartphone today. The utility is limited though, and if iPhone 19 still has arbitrary application memory requirements then I think it's safe to say they won't be using the most complete models. At the end of the day, OpenAI's "huge-ass LLM as a service" will probably be relevant longer than you think it will be. Local inferencing might be able to do simpler stuff (eg. compose assistant speech) but this is already possible with pruned models and CPU acceleration. The bull mindset is fun to watch unfold (especially here on HN) but I think people should temper their expectations.
  • endisneigh 4y ago
    I never said anything about openai not being relevant. the point is scale and good enough. gpt-4 is already good enough for most reasonable use cases, notably siri optimization. 5 years is a long time. 5 years ago llms didn't even exist.
    • shmoogy 4y ago
      Gpt3.5-turbo runs laps around siri and is cheap, surely Apple will acquire talent to build something that does remote queries and falls back to local for the next 1-2 years while they figure out on device accelerated LLMs
      • scarface74 4y ago
        People keep saying this. But it’s only half the problem to process natural language in text form. How well would ChatGPT do at understanding Spanish from a non native speaker with a southern US accent? (Raises hand). I’ve yet to see a speech to text system that does well at understanding speech by non native speakers or even native English speakers with a heavy regional accent.
        • yazaddaruvala 4y ago
          We have yet to see any Large Speech Models or Large Multimodal Models that use speech and text as inputs. It’s only the being but the general process to build this stuff now exists and the value proposition is crystal clear.
• deleted 4y ago
  [deleted]
• armchairhacker 4y ago
  This would be awesome. I don't like having to rely on closed-source cloud services for LLMs. But what incentive does Big Tech have? Especially considering that presumably they could monetize the cloud services easier. It's not like most consumers care either. I expect smaller companies like Stable Diffusion and grant/govt-funded research to bring cheap, local inference. There's definitely a lot of demand. But is there more demand than cloud-based services, so that it's economically viable for the biggest companies (Apple)?
  • endisneigh 4y ago
    i'd say apple is one of the few companies that could charge people for an "AI embedded" 128GB module, allow people to pay for a subscription in order to "access", yet have the inference happen locally. one hypothetical is that you need an icloud subscription, a token is retrieved from apple, the token "unlocks" your AI module, on the phone and allows you to do the inference. in this way apple could charge monthly for this and claim that the inference happens locally. sadly in this was it was similar to the whole csam debacle apple is also one of the few companies that could realistically get manufacturers to massively produce a hypothetical "AI model chip" that has the model on device, at a quantity that would make it realistic to pay for in a hypothetical iPhone 19 Pro model.
    • brookst 4y ago
      What other services does Apple charge a subscription fee for despite having no opex?
      • endisneigh 4y ago
        apple fitness I imagine has a pretty small opex (sure they pay trainers, but that doesn't scale linearly with subscriptions. the videos themselves are limited in scope and could be cached pretty trivially. the actual fitness tracking happens on device), same with apple arcade which I highly doubt has any additional expensive over the app store in general. I can see it - "Siri+", pay $5 a month for fine tuned model upgrades straight to your device and remote fallback. local inference available for Pro devices only.
• varispeed 4y ago
  Isn't that quite risky business - if someone manages to leak the model, anyone could run it for free? I can imagine similar for iPhone on the edge, when someone manages to decrypt the model it will be free to grab for anyone, unless there is going to be some proprietary thing going on only available on Apple Silicon that is undocumented.
  • endisneigh 4y ago
    I mean this risk is already there for all iphones for unlock.
• freediver 4y ago
  18 months.
• api 4y ago
  Apple M series chips are already some of the best chips for running your own models locally, especially if you want a laptop or small form factor machine instead of a big gaming desktop with a big GPU. Apple is really getting a niche here for machines to run models locally. That’s pretty powerful.
  • smoldesu 4y ago
    > Apple is really getting a niche here for machines to run models locally. I... don't agree. They have an acceleration API and a large install-base, but all of these models have run just fine on traditional hardware. GPT-Neo, Stable Diffusion and LLaMa can all be used and accelerated without Apple Silicon. Powerful, maybe. But not really unique, just putting up table stakes.
  • itg 4y ago
    I would not say they are the best chips if you want to run locally. They are ok although a bit slow compared to desktop gpus for inference.
    • jazzyjackson 4y ago
      What desktop GPUs would you recommend for the home user with access to 20+ GB of VRAM? I can get a Mac Studio with 128GB of shared RAM for about 5k.
      • lostmsu 4y ago
        What's the point of shared RAM exactly? 2x3090 ($2000) + 128GB RAM ($300) to cache weights would be much faster. Probably around 10-20x.
      • dragonwriter 4y ago
        > What desktop GPUs would you recommend for the home user with access to 20+ GB of VRAM? I think the usual current model would be the 24GB Nvidia RTX 4090.
• 8f2ab37a-ed6c 4y ago
  OpenAI IS big tech at this point given the massive financial and infrastructure investment from Microsoft. I think we're well past the small underdog narrative.
• s1k3s 4y ago
  Isn't GPT so complex that it requires hundreds of GB of ram to be used? How's it going to run on iphone?
  • endisneigh 4y ago
    yes, there are billions of parameters necessary. but large language models only came out about 5 years ago. I'm confident 5 years from now the parameters necessary to get gpt-4 performance will be decreased orders of magnitude. at the very least, even if that's not the case, inference will be drastically less gpu heavy by then I suspect.
    • s1k3s 4y ago
      Wait, so there's a way to make a model as smart as GPT but with less parameters? Isn't that why it's so good?
      • endisneigh 4y ago
        that's a complicated question to answer. what I'd say is that more parameters makes the model more robust, but there are diminishing returns. optimizations are under way
      • month13 4y ago
        This is an older paper, but DeepMind alleges in their Chinchilla paper that far better performance can be extracted with fewer parameters; quote "We find that current large language models are significantly under-trained, a consequence of the recent focus on scaling language models whilst keeping the amount of training data constant." It's difficult to evaluate a LLM's performance as it's all qualitative, but Meta's LLaMA has been doing quite well, at even 13B parameters.
        • astrange 4y ago
          Chinchilla is aimed at finding a cost-performance tradeoff as well, not the optimal amount of training. If cost is no barrier because it'll be used forever, then probably there's no amount of training that's good enough.
• guestbest 4y ago
  Apple will never have their tech run locally for AI anymore than they allow Siri to search the internet offline or while driving
  • norgie 4y ago
    How would anyone search the internet offline?
    • jazzyjackson 4y ago
      Simple, you make a local copy of the internet.
    • guestbest 4y ago
      You can’t . That’s the joke. If you use car play and ask Siri to find a local destination or a celebrity birthday, the functionality will be refused since the driver is assumed to be operating the car and thus blocks internet search, even though it will search in Apple Maps which obviously isn’t cached on the device
• ibejoeb 4y ago
  OpenAI's relationship with Microsoft pretty much makes it big tech. I think you have a point as long Apple keeps innovating its hardware, but it's not really a David vs Goliath situation.
  • brokencode 4y ago
    Nvidia is also no David to Apple’s Goliath. They are both Goliaths who will most likely be battling to build the best hardware for LLMs in the near future.
    • highwaylights 4y ago
      It’s an exciting time. I think people are very prematurely counting AMD and even Intel out of this race considering the pace of change.
      • thallium205 4y ago
        Intel is getting way too comfy soaking in government subsidies. It’s definitely out on the innovation front.
    • gitfan86 4y ago
      NVDA has the potential to hit huge profit growth in the short term, but not long term
      • btotes 4y ago
        What makes you say that?
        • gitfan86 4y ago
          There may be a short term panic to get position of CUDA GPUs which will allow NVDA to increase their prices a lot. But then other chip makers and software creator will build alternatives to NVDA
• guestbest 4y ago
  Apple will never have their tech run locally for AI anymore than they allow Siri to search the internet offline or while driving. Apple will only build the client as an html wrapping or inside an app but it will always run on apple servers
  • Pulcinella 4y ago
    Certain Siri features (and other AI features like photo subject identification) already run locally.
• bradDonniger 4y ago
  They under estimate chip makers plans to embed AI in silicon. Chips intended for launch in 5-6 years are in planning stages right now. Apple, nVidia, and Intel could bring serious hurt to software companies in the next 5-10. Open source purists will weep but really most people do not care, and tech should not merely serve the dedicated.
  • kanyethegreat 4y ago
    > Apple, nVidia, and Intel could bring serious hurt to software companies in the next 5-10. What does that mean? What software companies?
    • bradDonniger 4y ago
      Cloud based AI https://news.ycombinator.com/item?id=35283852 https://news.ycombinator.com/item?id=35283852
• l33tman 4y ago
  You can already run stuff like it on mobiles, for example alpaca.cpp with Llama models run fine on the CPU on any device that has a few gigs of RAM. Getting to even larger models using weight quantization, distilling etc it's not 5 years, it's 5 weeks or maybe 5 months at most.. Btw transformers are really simple and the optimisations making them run fast on CPUs have come a long way. I don't know the M1/M2 benchmarks for this, but many CPUs for edge devices have NN accelerators on the silicon that can run this, it's the pairing of the accelerator and gigs of RAM that is the key.
• dev_tty01 4y ago
  I'm expecting the Mac Pro to have AI/ML massive GPU modules and/or specialized ML dedicated compute available as add-ons for in-house training. For a not-so-small fee of course. They have all the pieces, it just needs to be brought to bear on the ML training market.
  • brookst 4y ago
    UMA is the secret. A 256GB Mac Pro can dedicate almost all of that to AI. Their GPUs are the weak spot. The Nvidia 4080 is 2.5x faster than the M2, and the A100 is 15 times faster.
    • ohgodplsno 4y ago
      The 4080 is _25x_ faster than the M2 on pure fp32 (which is what most GPUs are doing most of the time). Apple compared the M2 to the laptop 4080, using numbers heavily biased to them (running a 4080 at 10W does tend to make it not perform, yes). Not a single benchmark in the world has supported Apple's claim that the GPU in the M2 is that powerful. It's just yet another cute embedded GPU that does the job, but nothing more. It's made to push out 8K frames really fast, which it does because of UMA, but want demanding task will have it be eaten alive by any real GPU.
      • brookst 4y ago
        ML inference is not generally FP32 anymore. I was going off of the TOPs numbers for ML from a few sources, which generally agree M2 is about 22 TOPS and 4080 (desktop) is about 50. But in any event, yes, that was my point. UMA is a huge advantage, the GPU itself is too weak to be serious. But it’s a lot easier to drop a dramatically beefier GPU into a new design than it is to update the entire platform for UMA. Apple has a huge opportunity here… whether tbey pursue it or not remains to be seen.
        • smoldesu 4y ago
          > whether tbey pursue it or not remains to be seen. Pursue what though? UMA is cool, but kinda meaningless if the majority of Macbooks are min-spec. That leaves you with 4-5gb of VRAM, assuming you've left nothing open. What is Apple going to do with that UMA that other manufacturers cannot? It's certainly nice that 128gb Macs exist for models that might be too big to otherwise load into memory. It's useless for production inferencing though, and I struggle to imagine the "opportunities" they're missing out on here.
• 0xDEF 4y ago
  >openai is doing great work and is serious competition, but I think many underestimate big tech OpenAI is 49% owned by Microsoft. OpenAI is big tech.
• j45 4y ago
  This sounds great, but why can’t Siri still not understand me. Maybe Apple has the ultimate
• guardian5x 4y ago
  Yea, probably right after they released the hyped Apple Car. It's probably just month away now!
  • anonymouse008 4y ago
    I'm convinced there are quite a few products for the 'Patagonia vest crowd'* *I used to be one in a Sales and Trading role and fully believe a lot of rumors are to keep the finance analysts happy.
• option 4y ago
  I’ll give it 2 years max to reach ChatGPT like capabilities running locally on latest iPhones and Macs. Lets hope Apple doesn’t screw this up.
  • lern_too_spel 4y ago
    That's skating where the puck is. By that point, we'll be using trillion parameter models in the cloud that make ChatGPT look like a toy.
• 1-6 4y ago
  Live translation on AirPods, here we go!
  • andsoitis 4y ago
    a babelfish in your ear
• schrodinger 4y ago
  Agreed. Apple is fantastic at waiting for the moment technology to be mature enough to be useful to everyday people and jumping on it. They have a fantastic CPU department that no one else can match, and certainly must have a skunkworks AI team. They’ll let everyone else make the mistakes, learn from them, and release a great product.
• lanza 4y ago
  OpenAI is big tech. Also, Apple has nowhere near the AI/ML staff that the rest of the tech giants have.
  • bjtitus 4y ago
    Big tech that already has comparable open source models that can run on consumer hardware?
• bitL 4y ago
  That's highly unlikely. I think most people can't fathom how huge is the scale of GPT-4 and what computing power it requires. Even if 10x optimizations, 10x performance and 10x memory (highly unlikely in 5 years) that's not going to be sufficient to run it locally.
  • yieldcrv 4y ago
    I think people are overestimating how many parameters GPT-4 is really trained on. While also overestimating how many parameters anybody really needs after fine tuning. The market will find the sweet spot. Right now everyone’s tinkering with the 7B parameter LLM and then going to move up to the 65B one once they've refined the process. I think its fiction that anybody really needs a 10 trillion parameter LLM at all. It will be completely niche.
    • bitL 4y ago
      Fine-tuning is a solved problem using transformer adapters which are fast, small and match regular fine-tuning, so that's not an issue. As for smaller models, their usefulness will quickly wane when they will be seen as producing trivial gibberish in the light of cloud models available in 5 years.
• yieldcrv 4y ago
  I would say 1 or 2 years at the OS level Basically the curtain has been drawn and shows that current LLM’s are just very inefficient and will be optimized in weeks. Whatever improvements Apple’s Neural chip offers just needs more RAM closer to it, which will likely come at the next hardware fresh where this year its probably too late, while whatever is released at the end of 2024 will be good enough.
• ThorsBane 4y ago
  This will be the AI killer app for whoever pulls it off first. GPT-4 or higher performance on a device as nice as an iPhone...will create so many new capabilities for smart phones. It's going to be an amazing time for tech in the near future and beyond.
  • calvinmorrison 4y ago
    why does this matter when we have cloud computing and high speed mobile internet? why does it need to be local
    • ThorsBane 4y ago
      Because it's more seamless and then integrates into the Apple hardware ecosystem due to AirDrop and other integrations.
      • ThorsBane 4y ago
        And also it represents cool and significant hardware advances. That’s always sexy, and on-device AI will expand device capabilities. Imagine Google Sonar on Pixel phone with Pixel ear buds and more sensors and ChatGPT onboard.
    • deleted 4y ago
      [deleted]
• philip1209 4y ago
  I agree that you're going to see Apple start to push for on-device AI. OpenAI promotes thin clients - run everything in the data center. Apple has been a proponent of thick clients - run everything as possible locally. So, it make sense that Apple will start to promote local LLM support. That's why they pushed Stable Diffusion support so quickly: https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon https://machinelearning.apple.com/research/stable-diffusion-...
• doctoboggan 4y ago
  Apple is such a cautious company in many respects. Do you think they have the ability to ship a largely unpredictable piece of software like an LLM?
• abraxas 4y ago
  Greed is a fabulous motivator but it can't surpass the laws of physics. This is probably why we don't have a cure for cancer or room temperature superconductors or jurassic parks. It may be so that scaling down a language model like GPT 4 is not possible on hardware systems orders of magnitude smaller than the one used by OpenAI. I'm not saying it's impossible but it's fallacious to just assume outright that it's inevitable because market forces. For all we know it could turn out that the only way to get a gpt in a pocket format is through some form of analog chips that must be individually trained that Hinton aptly described as an era of mortal computing.
  • alwayslikethis 4y ago
    There is nothing impossible about a Jurassic park in terms of laws of physics though. It's a matter of getting the right materials to recreate dinosaurs. Same goes for cancer cures. I'm not sure if there are physical limitations preventing a room temp superconductor.
    • kergonath 4y ago
      > I'm not sure if there are physical limitations preventing a room temp superconductor. Not really. We just know that it’s very uncommon at best. The limit is material dependent and we already have superconductors with more than one order of magnitude difference in their critical temperature (e.g. ~4 K vs ~40 K; YBCO, which is widely studied, is at 90 K). It is not inconceivable that we could come up with some fancy material with a critical temperature three times as high again. There are several laboratories with good money working on it.
• levidos 4y ago
  Why would they want to give out their trusty LLMs when they can just hide it behind an API, and making sure you'll be dependent on them forever
  • quickthrower2 4y ago
    Running locally and “give you” are two very different things. Financially you paid for the phone. From a lock in point of view the chip will be proprietary and very hard to reverse engineer. They can indeed hide it behind a local API that apps that they can ban at will would need to use.
  • dragonwriter 4y ago
    > Why would they want to give out their trusty LLMs OpenAI won’t want to; open source competitors already are, and they will keep getting better. The more of a lead OpenAI has over commercial competitors, the more incentive those commercial competitors will have to back open source options.
• todd3834 4y ago
  I’m a huge Apple fanboy, I even moved my family up to the Bay Area to work for them for a time. Hopefully you’re right but if Siri is any indicator for some reason Apple can’t seem to crack AI at the same level as OpenAI has proven to be able to.
• deleted 4y ago
  [deleted]
• sparrc 4y ago
  I'm not so sure. When is the last time apple came out something groundbreaking? Has there been anything since Tim Cook took over? Siri is still extremely basic and barely usable for anything besides starting a kitchen timer.
  • Dalewyn 4y ago
    Apple Silicon.
  • inferyes 4y ago
    > I'm not so sure. When is the last time apple came out something groundbreaking? M1 chips?
    • sparrc 4y ago
      An ARM cpu is groundbreaking? I don't think anyone outside the tech world has even noticed.
      • rorytbyrne 4y ago
        Consumer blogs have been gushing over Macbook Air M1 for years, and most of them say not to even bother getting a Pro because it's so good.
      • pb7 4y ago
        Name something that you find groundbreaking and I too can reduce it down to something mundane. Leaps in performance across all metrics in an existing thing is groundbreaking, especially when it’s just the beginning. On a practical level, it feels pretty groundbreaking to me when I go back to use my previously top of the line 16” MBP from a year before. I suspect you haven’t had the pleasure of using an M-series computer.
• atleastoptimal 4y ago
  Local inference is a pipe dream. There's no way any of these companies will allow consumers to interact with their flagship models on anything but a black box api.
  • kergonath 4y ago
    The black box could well be a local library. Functionally, what would be the difference between using something like AIKit and using a REST API over the network? Local has a lot of advantages as well (latency, privacy, etc).
• anshumankmr 4y ago
  Something like FLAN T5 can't run without top of the line VMs and GPT 4 is an order of magnitude better than that. How will it run on an iPhone, no matter how good it gets.
• thund 4y ago
  it will actually play out differently: Apple will pay Open AI to run their models on Apple Silicon. And Apple will guarantee the model is protected using hw keys. Jail broken phones won’t have access to the local AI
  • whywhywhywhy 4y ago
    I feel the sun is about to set on general purpose computing. Everything from here on out is going towards being silicon locked.
• illegalmemory 4y ago
  I also believe this is going to the case. The past decade apple has positioned itself as a privacy focused + good hardware company. The next is going to be offline AI + privacy + good hardware & I am already sold to the idea.
• fullsend 4y ago
  Definitely possible. A cool idea. But that’s also a lot of credit for “big tech”. Meta, Google, Netflix, and others have produced almost nothing cool for a decade. Apple is one of the few behemoths seemingly capable of actually getting any real work done, so you’re right they’ll be a big contender. Smaller players have less capital, but a lot less blockers too. They can take real risks, move faster, and don’t have all the bottlenecks that come with having 50,000 employees.
• random_kris 4y ago
  5 years ? I'd say 1 or 2. Or this September already judging by the rate of change recently
• Cthulhu_ 4y ago
  Definitely, for some years now the iPhone has had AI stuff on board to categorize and organize your photos (recognising people, creatures, "things" etc), all on board and built into the device - unlike Google that requires you to upload them first. It's not a stretch at all to have more functionality like Siri run on-device.
• christkv 4y ago
  One step at a time towards the "family AI's" of Neuromancer.
• hardware2win 4y ago
  OpenAI and MSFT couldnt figure it out and are burning milions of USD on compute, but Apple will make it run on phone in 5 years? Crazy bet, what makes you think that? You cannot optimize infinitely. Raytracing probably had decades of ppl trying to make it run fast and yet even today you need strong hardware
  • aunty_helen 4y ago
    Have a look at what the opensource community is up to. They're reducing models down to the meer billions of nodes and then using chatgpt to train their model accuracy. The results are suprising. The reason openai have had to rush out plugins is due to software like langchain coming in at meteoric speed. Things are moving on a day to day basis in the ai sphere at the moment.
• aunty_helen 4y ago
  Honestly, I've looked at shorting MS due to openai. Their technology (LLMs), or the secret sauce, can easily be stolen just by the process of putting that tech out there. Have a look at Alpaca, FB made it, someone leaked the weights and now there's a dataset of training it for only a few hundred dollars that can beat openai at its best. Not everyone needs to employ a PhD for doing customer service, in the same way not everyone needs GPT5 for answering support queries. Their business model is leaking away from them.
  • anukin 4y ago
    TBH alpaca is nowhere near to chatgpt or even gpt models in its capability. I have been playing around with lama for a while now and even chain of prompts, the quality of answers are really bad. I had thought that Facebook would be having the ability and competence to produce a LLM which can outclass open ai
• mirekrusin 4y ago
  When you'll be able to run gpt4 on iphone 19, there will be gpt27 that doesn't run on it.
• danieldrehmer 4y ago
  Why 5 years? I'm running a totally usable 13b parameters llama model in my macbook air, which seems to give outputs equivalent to what I was getting from GPT3 in June 2022. How much more hardware would it really be needed for GPT-4 level outputs natively? Perhaps software optimizations alone could do most of the trick.
• whywhywhywhy 4y ago
  There is no benefit to OpenAI not needing a data centre for their tech. The only ways they have to lock their tech up is safety and ethics theatre and the hardware budget required to run it. if you work at OpenAI and you’re optimizing anything you’re not doing your job right.