7 ms·
Apple to Power AI Features with M2 Ultra Servers
- bsaul 2y agoWonder what kind of tech they’ll be using on the server side to develop all those services.
- mousetree 2y agoSwift perhaps
- zitterbewegung 2y agoTensorflow and PyTorch have been ported to use the accelerators on the M series of chips and they could just be deploying Mac Pros that are rack mounted. So Python.
- amelius 2y agoWill they be using the ports that have been painstakingly made by open-source enthusiasts without having access to official Apple documentation?
- kristjansson 2y agoI’m pretty sure you’re thinking about the work on the open source GPU driver. AFAIK Apple contributed the MPS backend for pytorch and tensorflow-metal PluggableDevice.
- coldtea 2y agoThey often work with the community in helping the ports for a lot of these projects, it's just not publicized as much (and often their engineers aren't allowed to talk about it). E.g. when Apple Silicon was released there were people in various FOSS projects who worked with Apple assistance for porting. Here's an example that comes to mind: "In collaboration with the Metal engineering team at Apple, we are excited to announce support for GPU-accelerated PyTorch training on Mac." https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/ https://pytorch.org/blog/introducing-accelerated-pytorch-tra...
- bitcrusher 2y agoIt's been a few years but when I was there, Apple had an almost frothingly EXTREME policy against using ANYTHING from Google, most especially Tensorflow. Things may have changed in the last few years but I would not be surprised if that were still the case.
- zitterbewegung 2y agohttps://developer.apple.com/metal/tensorflow-plugin/ https://developer.apple.com/metal/tensorflow-plugin/
- re-thc 2y agoiAI
- cayleyh 2y agoSome likely candidates are the existing open source tools their ML team has released, most of which either use PyTorch or are usable via PyTorch: - CoreNet, a training toolchain developed by their ML teams: https://github.com/apple/corenet https://github.com/apple/corenet. It's built on PyTorch, but uses MLX - MLX, Apples internal array processing framework which is C/C++/Swift (https://pypi.org/project/mlx/ https://pypi.org/project/mlx/). - CoreML tools, their python package for accessing the lower-level CoreML APIs (https://github.com/apple/coremltools https://github.com/apple/coremltools)
- oldpersonintx 2y ago[dead]
- dralley 2y agoRunning what operating system?
- the_panopticon 2y agoMaybe baremetal BSD 4.4? That's the codebase for the Unix personality layer of OSX (which is Mach as ukernel + a Unix layer on top). Or FreeBSD like Netflix uses for their first-party servers?
- deleted 2y ago[deleted]
- derefr 2y agoIIRC Apple has had a history of using NetBSD for server/infra systems, especially those demanding high network throughput and stability. Examples: • their ObjC WebObjects-based web backends (such as the iTunes Music Store backend — which might still be ObjC-based to this day? Anyone know?) ran on NetBSD servers • the firmware in Airport routers was NetBSD I wouldn't be surprised if there’s an Apple Silicon build of NetBSD, created for internal use at Apple. (Though I also wouldn't be surprised, if they've tried to "converge efforts" since then, and have somehow stuck a Darwin userland on top of a NetBSD kernel.) I would be genuinely surprised, though, if the infra folk at Apple trust the XNU kernel's network stack enough at this point to want to use plain XNU/Darwin for their servers! --- As an aside, I've always been fascinated by Apple's approach to backend/server technologies, compared to other bigcorps. Despite not being a "UNIX shop" per se, Apple seems to have acquired some engineers somewhere along the way who have a deep understanding of the "UNIX way" to build stuff, and who carry what might even be called "legacy ideas" about system architecture. You can get a peek into this approach, by prying into the insides of the old Server.app. Sort of like how XCode.app has a BSD buildroot inside it, with the base OS shipping with stub binaries that call inside it when it's available; Server.app does the same, but with an extended BSD userland containing regular old BSD daemons. Besides being a wrapper for this BSD-userland payload, Server.app itself was just a configuration wizard and state-converger for a set of plain-old config files, living in a virtual /etc dir (regenerated from canonial plist files), that enabled and drove these venerable daemons to do their thing. The "Websites" feature was just Apache, and the "Wiki", "Calendar/Contacts", etc features were just symlink-managed PHP-FCGI plugins for that Apache instance. The DNS was just BIND. The VPN feature was mostly racoond[1]. It's exactly what you'd expect from e.g. cPanel on Linux, with no sense of anything "Apple-y" going on. [1] https://www.netbsd.org/docs/network/ipsec/rasvpn.html https://www.netbsd.org/docs/network/ipsec/rasvpn.html This UNIX-y approach also seemingly extends to Apple's use of networking protocols. Even while Apple was giving users a proprietary SMB-alike protocol (AFP) to use for local-network file sharing, their internal approach was different: macOS, to this day, ships with a pre-configured "auto-home" mobile user profile feature, that expects to talk not to an AFP server, or even an SMB server, but rather to an NFS(!!) server. (You could even set this up for yourself, in the Server.app days, given sufficient understanding of Apple's OpenDirectory + how it integrates with BSD Kerberos.)
- marban 2y agoRunning on Snow Leopard Server?
- boesboes 2y agothose were the days.. I feel Snow Leopard was peak OS X. Clean, lean, no bullshit. It seemed magical how well everything worked. To be fair though, current versions are probably 'better' overall, I'm just annoyed with all the new stuff and crappy apps haha
- TheAmazingRace 2y agoI concur. Apple really needs to get back to their roots and come out with a bugfixed macOS release without all the extra gunk. Perhaps even make it an LTS release as a last hurrah for Intel Macs.
- jwells89 2y agoIt was great, but when I use it there’s little bits of accumulated polish here and there that I feel the absence of (e.g. inertial scrolling, which as it turns out is something I use heavily). Would love it if someone were to build a desktop that uses 10.4/10.5/10.6 as its design basis and then adds that missing polish.
- bearjaws 2y agoProbably as a stop gap solution until they make a true AI-centric chip. Get's all the moving parts into place for when they have a competing solution to Nvidia / Google Tensor.
- ChildOfChaos 2y agoThe real problem is the amount of Ram they include in the processors, Apple's insistence on keeping 8gb as the base is going to hold them back here.
- jmull 2y agoThe amount of ram on their lower end configurations really has nothing to do with this.
- ChildOfChaos 2y agoI'm not sure that a MacBook Pro at £1,899.00 should be a 'lower end configuration', if we are talking bout running AI locally and Apple want's to get in this game, they can't expect everyone to buy an extremely high end machine because their Macbook 'PRO' only has 8gb of RAM.
- stetrain 2y agoWhat does the base 8GB RAM option have to do with how Apple will spec their servers? The M2 Ultra supports up to 196GB of RAM in the products Apple offers it in.
- ChildOfChaos 2y agoI was responding to a comment that directly said this was a stop gap before they made a real AI centric chip, in that context, the chips will need more ram, until they do, they will be offloading a lot of stuff to the server. The Ultra may support 192gb but that also comes at an insane cost, if AI is going to become mainstream and Apple want to do this on device, the base models are going to need more Ram, processing power on these systems is not the bottleneck, the RAM is.
- iAkashPaul 2y agoSummarisation & SDXL run just fine on M1 & above Macs. Probably the video editing features like Google/Pixel might need cloud compute.
- gigel82 2y agoLOL, you can get 5x - 10x computational power (including AI if needed) for the same price from other vendors. Unless they overshot production and have stock of unsold M2s they're willing to sell at 90% discount.
- fsociety 2y agoApple doesn’t pay consumer prices for their own chips.
- IshKebab 2y agoSure but they also don't have a profit margin of like 800%. And there's power consumption to think about. I expect it's about control and Apple's philosophy that they have to do everything themselves.
- LordKeren 2y agoApple is not becoming a third party cloud service provider. This is hardware for their own internal uses
- swatcoder 2y agoThey don't pay retail for their own vertically integrated products, they likely would have inventory surplus as new retail products move to other chips, and they mature production pipelines primed to efficiently squeeze out additional volume for a new application like this. Your comment doesn't make much sense to me.
- cayleyh 2y agoI wondered about this too. One thing that did make sense to me was that these are for supporting features as-is on older devices by offloading on-device workloads to the cloud. If you're releasing a bunch of new ML-powered features that are setup to run on-device, you have to have a cutoff for what generation of device is required. Apple has indicated with recent OpenELM model releases that they're targeting different levels of device capability. But you can only go so far. For older devices, you can offload those tasks to the cloud, but then you have to maintain 2 different software stacks and there are potential compatibility issues. If you have a bunch of M2Us in the cloud though, you have the exact same hardware that you're developing on-device for. If you package up the core software components as frameworks, you can run on device or in the cloud, and it MAYBE simplifies keeping the results and output in sync between the two. Otherwise, it makes no sense from a pure performance perspective vs. just deploying racks of H100s...
- Evidlo 2y agoWaiting for Apple to make MK Ultra servers
- chefkd 2y agounderrated comment :) bold of you to assume they don't have them already only 6 countries have a higher GDP than apple lol if anyone from Apple is in these threads please sign me up for the beta although i think the first round pretty much knocked me & my family out
- syrgian 2y agoYou are comparing market cap to GDP. You should compare revenue to GDP. Still not a perfect comparison, but much much better. Apple's revenue: 380B. Iran sits at position 40 with 380B GDP in 2022.
- chefkd 2y agoahhh makes sense thank you for catching that Iran keeps popping up all over HN past few days haha still kind of crazy to think Iran and it's entire oil reserves that power idk how many households produce the same amount of revenue as 175k people
- throwaway11460 2y agoThat oil is sold unprocessed and won't be powering anything unless it goes through a fairly expensive processing and logistics chain. Iran is also extremely inefficient and unproductive - it's a state after all. Similarly, African raw materials form the core of many computers and yet Africa is still poor - because the raw material itself isn't that useful.
- tahoeskibum 2y agoI am still amazed that I can run a local instance of LLama3 on my 3-year old M1 MacBook Air. It is a bit slower than ChatGPT online, but nice to have a GPT-3.5 level LLM running locally!
- deleted 2y ago[deleted]
- jamiek88 2y agoI kinda checked out of LLM news as the hype cycle had got annoying and I had no idea this is possible. I have an M1 Pro and am excited to try it.
- duxup 2y agoEat your own dog food. Whatever they learn / develop there could pay dividend elsewhere. Seems logical. Got some questions or ideas, you know a guy who knows those chips.
- tiffanyh 2y agoECC I wonder if that means Apple will bring ECC to Apple Silicon.
- yjftsjthsd-h 2y agoIf Apple is going to build servers, it sure would be nice if they went the extra half-step to also sell them so people could stop with the weird rack-mounted mac minis. I guess https://www.apple.com/shop/buy-mac/mac-pro/rack https://www.apple.com/shop/buy-mac/mac-pro/rack is a step towards that, but "real" server hardware would be better.
- sciencesama 2y agoNow they will realize 8gb won’t cut it
- fragmede 2y agoThe fascinating decision someone at Apple has to make is how to allocate the production of chips. Every M2 Ultra chip that goes into a server is a chip that can't go into a consumer device, and vice versa. They're the only source for M2 Ultra chips, so they can't go somewhere else to get more, so they have to decide which division gets what.