9 ms·
Apple approves driver that lets Nvidia eGPUs work with Arm Macs
https://xcancel.com/__tinygrad__/status/2039213719155310736 https://xcancel.com/__tinygrad__/status/2039213719155310736
- bigyabai 6mo agoThe opportunity cost of Apple refusing to sign Nvidia's OEM AArch64 drivers is probably reaching the trillion-dollar mark, now that Nvidia and ARM have their own server hardware.
- chuckadams 6mo agoApple got out of the server game long before they adopted aarch64, so that's a trillion worth of server hardware they never would have sold anyway. And probably not actually a trillion.
- bigyabai 6mo agoApple was the only one stopping themselves from getting back in. It's not like the Mac is a trillion-dollar market segment to begin with.
- QuantumNomad_ 6mo agoAlmost everyone including myself had MacBook Pros at my last place of work. If Apple was in the high-end server market, I see no reason why the company I was working for would not be running macOS on Apple hardware as servers, instead of the fleet of Linux based servers they had.
- deleted 6mo ago[deleted]
- bigyabai 6mo agoWhy wait? You can go run macOS as a server right now. It will take you a few hours to get Docker working, and disable mdworker_shared() and turn off SIP, and then install a package manager/XCode utilities, and finally configure macOS to run as a headless UNIX box, but it's attainable. Despite how easy Apple makes it, nobody is really using Macs as a server in production. Apple[0] is not using them as a server in production. They would need a radically different strategy to replace Linux, because their efforts on macOS still haven't replaced Windows. [0] https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-reportedly-sitting-unused-on-warehouse-shelves-due-to-low-apple-intelligence-usage/ https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep...
- varispeed 6mo agoUSD starts sounding more and more like meaningless tokens. Billion here, trillion there. I still have 100 trillion Zimbabwean dollars somewhere.
- altairprime 6mo agoFeels like that here in the U.S., too.
- wmf 6mo agoPretty misleading. This driver is only for compute not graphics.
- polotics 6mo agoAs a sizable share of the market is going to want to use this for local LLMs, I do not think this is that misleading.
- bigyabai 6mo agoMost people I know are not using TinyGrad for inference, but CUDA or Vulkan (neither of which are provided here).
- manmal 6mo agoGraphics was not what came to mind when I saw the headline.
- eoskx 6mo agoInteresting, but cannot run CUDA or more to the point `nvidia-smi`.
- embedding-shape 6mo agoWell, to be fair, the whole shebang is from a completely different company, that have their own ML library and such, so that isn't that surprising. Although I agree that some CUDA shim or similar would be a lot more interesting, still getting to the place of running inference and training with your very own library is pretty dope already.
- arjie 6mo agoWoah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?
- manmal 6mo agoMaybe I’m lacking imagination. But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? The interconnect isn’t powerful enough to change layers on the GPU rapidly, I guess?
- arjie 6mo agoMy Mini is actually the smallest model so it actually has "small but slow VRAM" (haha!) so the reason I want the GPU for are the smaller Gemmas or Qwens. Realistically, I'll probably run on an RTX 6000 Pro but this might be fun for home.
- zozbot234 6mo ago> But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? It would work just like a discrete GPU when doing CPU+GPU inference: you'd run a few shared layers on the discrete GPU and place the rest in unified memory. You'd want to minimize CPU/GPU transfers even more than usual, since a Thunderbolt connection only gives you equivalent throughput to PCIe 4.0 x4.
- brcmthrowaway 6mo agoWhat are the limitations of USB4/Thunderbolt compared with a regular PCIe slot?
- embedding-shape 6mo agoWell, for starters, PCIe 5.0 x16 would do something like about 60 GB/s each way, while Thunderbolt 4 does 4 GB/s each way, TB 5 does 8 GB/s each way. If you don't actually hit the bandwidth limits, it obviously matters less. Whether you'd notice a large difference would depends heavily on the type of workload.
- givinguflac 6mo agoI think you missed a zero, TB5 does 80GB/s.
- Tepix 6mo agoNo. It does 80Gbps. https://www.convertunits.com/from/Gbps/to/GB/s https://www.convertunits.com/from/Gbps/to/GB/s
- givinguflac 6mo agoDerp, didn’t read closely enough. Thanks
- mch17 6mo agoNo, it does 80 Gb/s. With encoding loss it’s closer to 8GB/s
- deleted 6mo ago[deleted]
- deleted 6mo ago[deleted]
- justincormack 6mo agoIt carries pcie, but only at x4. Thunderbolt 4 is pcie gen 3 and Thunderbolt 5 is pcie gen 4.
- frankc 6mo agoMy main thought is would this allow me to speed up prompt process for large MoE models? That is the real bottleneck for m3ultra. The tokens per second is pretty good.
- embedding-shape 6mo agotinygrad does have pretty neat support for sharding things across various devices relatively easy, that'd help. I'm guessing you'd hit the bandwidth ceiling transferring stuff back and forth though instead.
- bangonkeyboard 6mo agoI don't know how Apple has evaded regulatory scrutiny for their refusal to sign Nvidia's eGPU drivers since 2018.
- GeekyBear 6mo agoThe same way Google evaded regulatory scrutiny for refusing to allow a YouTube client for Windows Phone?
- bigyabai 6mo agoInternet Explorer Mobile is a YouTube client. You're describing a client-server disagreement when the user is talking about an entirely client-based conflict.
- realusername 6mo agoGoogle deployed custom code to actively block the clients so it went beyond just a disagreement
- bigyabai 6mo agoThat's normal behavior when your server is being reverse-engineered or abused. Video bandwidth is not free. Apple's decision is not constrained by server logic or ballooning costs, it is entirely a client-based policy to not sign CUDA drivers.
- GeekyBear 6mo ago> That's normal behavior when your server is being reverse-engineered or abused. Video bandwidth is not free. Microsoft rewrote their Windows Phone native client to pass through Google's ads. Google still blocked it. Was it normal behavior when Google blocked Amazon Fire devices from connecting to YouTube with a web browser during the Google/Amazon corporate spat? To be fair, Google did back down almost immediately when the tech press picked up on it. Not allowing a native client for your monopoly market share video service on Amazon devices while also blocking Amazon's web browser on those devices is making things a bit too obvious.
- qoez 6mo agoIdk why this doesn't link to the original source instead of this proxy source: https://x.com/__tinygrad__/status/2039213719155310736 https://x.com/__tinygrad__/status/2039213719155310736
- multjoy 6mo ago[flagged]
- TeMPOraL 6mo agoIsn't X usually the original source these days?
- Forgeties79 6mo agoIt’s often a secondary/tertiary source unless you’re looking for official statements.
- amatecha 6mo agoProbably because you can't actually read anything more than the initial post without getting a login-wall: "Join X now to read replies on this post." (Not to mention "X" is a trash site now)
- beepbooptheory 6mo agohttps://xcancel.com/__tinygrad__/status/2039213719155310736 https://xcancel.com/__tinygrad__/status/2039213719155310736
- collabs 6mo agoI don't go there unless I'm looking for something specific but I've found adding cancel at the end helps https://xcancel.com/__tinygrad__/status/2039213719155310736 https://xcancel.com/__tinygrad__/status/2039213719155310736
- seaal 6mo agoI mean, just setup redirector extension and never think about it again. Redirect: https://x.com/* https://x.com/* to: https://xcancel.com/$1 https://xcancel.com/$1
- the__alchemist 6mo agoI'm writing scientific software that has components (molecular dynamics) that are much faster on GPU. I'm using CUDA only, as it's the eaisiest to code for. I'd assumed this meant no-go on ARM Macs. Does this news make that false?
- wmf 6mo agoThis driver doesn't support CUDA.
- ksec 6mo agoThis comment should be pinned at the top.
- brcmthrowaway 6mo agoIsnt mlx a cuda translation later?
- wmf 6mo agoDoes tinygrad support MLX?
- superb_dev 6mo agoMy understanding is that MLX is Apple’s CUDA, so a CUDA translation layer would target MLX
- ykl 6mo agoNo, it’s not. MLX is Apple’s NumPy more or less.
- ykl 6mo agoNo, MLX is nothing like a Cuda translation layer at all. It’d be more accurate to describe MLX as a NumPy translation layer; it lets you write high level code dealing with NumPy style arrays and under the hood will use a Metal GPU or CUDA GPU for execution. It doesn’t translate existing CUDA code to run on non-CUDA devices.
- Keyframe 6mo agoSuch a shame both companies are big on vanity to make great things happen. Imagine where you could run Mac hardware with nvidia on linux. It's all there, and closed walls are what's not allowing it to happen. That's what we as customers lose when we forego control of what we purchase to those that sold us the goods.
- deepsun 6mo agoDon't purchase? I don't own any Apple devices, everything works fine.
- aljgz 6mo agoI don't understand the logic for downvotes. We vote with our wallets. When I could not update the Ram on my personal Dell machine I asked for a Frame.work in my new job. As my Intel based FW at work had thermal throttling problems, for my next personal purchase I got an AMD one. As Ubuntu had shady practices, I installed Fedora, as Gnome forced UX choices I did not want, I used KDE. As I wanted my machine to be even more stable I use an immutable spin. The machine I'm using now represents my choices and matches what matters to me, and works closer to perfectly than all my machines in the past And yes, I have worked with macs, and no, the UX and the entire tyranny in the Apple ecosystem was not something I could live with And yes, this machine is fast, predictable, a joy to work with and is a tool I control, not a tool to control me. If something happens to it, I can order the part with the same price that goes into a new machine, and keep using my laptop
- TheDong 6mo ago"We vote with our wallet, so don't complain" is a bad take in my opinion. Like, for phones, I want a phone which runs Linux, has NFC support, and also has iMessage so my friend who only communicates with blue-bubbles and will never message a green-bubble will still talk to me. I also want it to have regulatory approval in the country I live in so I can legally use it to make calls. Because apple has closed the iMessage ecosystem such that a linux phone can't use it, such a device is impossible. I cannot vote for it. As such, I will complain about every phone I own for the foreseeable future.
- vondur 6mo agoIf you could get Nvidia driver support on Mac’s I bet Apple would have sold more MacPro’s.
- ProllyInfamous 6mo agoIf unfamiliar: it is a big deal that AAPL & NVDA again have an official relationship. For well over the previous decade Apple has not allowed newer nVidia GPUs (by not allowing drivers). A seven year old GPU (e.g. VEGA64, RTX1080Ti) can still process more tokens/second than most Apple Silicon (particularly the lower-ends). As discussed elsewhere, Apple MAX/Ultra processors are best-suited for huge models (but are not as fast as e.g. RTX5090).
- bigyabai 6mo agoThis is not an official relationship, this is a third-party effort by tiny corp with no Nvidia involvement.
- ProllyInfamous 6mo agoFrom headline title: >>Apple approves... This is a big deal.
- dd_xplore 6mo agoWhy does Apple need to make the drivers in a walled garden? Atleast they should support major device categories with official drivers.
- embedding-shape 6mo ago> Why does Apple need to make the drivers in a walled garden? Isn't that the whole point of the walled garden, that they approve things? How could they aim and realize a walled garden without making things like that have to pass through them?
- GeekyBear 6mo ago> Why does Apple need to make the drivers in a walled garden? For the same reason that Microsoft requires Windows driver signing? Drivers run with root permissions.
- mschuster91 6mo ago> Why does Apple need to make the drivers in a walled garden? Because third party drivers usually are utter dogshit. That's how Apple managed to get double the battery life time even in the Intel era over comparable Windows based offerings.
- wtallis 6mo agoDoesn't Apple support the major standard device categories: NVMe, XHCI, AHCI, and such, like most operating systems do? The challenges are all for hardware that needs a vendor-specific driver instead of conforming to a standard driver interface (which doesn't always exist). Lots of those can be supported with userspace drivers, which can be supplied by third parties instead of needing to be written by Apple.
- mlfreeman 6mo agoI followed the instructions link and read the scripts...although the TinyGPU app is not in source form on GitHub, this looks to me like the GPU is passed into the Linux VM underneath to use the real driver and then somehow passed back out to the Mac (which might be what the TinyGrad team actually got approved). Or I could have totally misunderstood the role of Docker in this.
- gsnedders 6mo agohttps://docs.tinygrad.org/tinygpu/ https://docs.tinygrad.org/tinygpu/ are their docs, and https://github.com/tinygrad/tinygrad/tree/4d36366717aa9f17356379296e36b4e690cdd8c7/extra/usbgpu/tbgpu/installer/TinyGPUDriverExtension https://github.com/tinygrad/tinygrad/tree/4d36366717aa9f1735... is the actual (user space) driver. My read of everything is that they are using Docker for NVIDIA GPUs for the sake of "how do you compile code to target the GPU"; for AMD they're just compiling their own LLVM with the appropriate target on macOS.
- MeetRickAI 6mo ago[dead]
- MeetRickAI 6mo ago[dead]
- tensor-fusion 6mo ago[flagged]
- mort96 6mo agoI mean when it comes time to output the image from the GPU, I don't want to add a hundred milliseconds of network latency...
- whalesalad 6mo agoThis is re gpu for compute not graphics.
- mort96 6mo agoOh. Weird use for a graphics unit.
- lostlogin 6mo agoIt’s what’s driven nearly the entire AI boom.
- nkrisc 6mo agoUsing GPU for compute is nothing new or unusual these days, not for quite a while.
- userbinator 6mo agoI've heard it phrased thus: The "G" in "GPU" stands for "general-purpose".
- mort96 6mo agoNo, but its primary purpose remains graphics
- nkrisc 6mo ago
- MrArthegor 6mo agoA good technical project, but honestly useless in like 90% of scenarios. You want to use an NVidia GPU for LLM ? just buy a basic PC on second hand (the GPU is the primary cost anyway), you want to use Mac for good amount of VRAM ? Buy a Mac. With this proposed solution you have an half-backed system, the GPU is limited by the Thunderbolt port and you don’t have access to all of NVidia tool and library, and on other hand you have a system who doesn’t have the integration of native solution like MLX and a risk of breakage in future macOS update.
- tensor-fusion 6mo ago[flagged]
- bigyabai 6mo ago> same PyTorch/CUDA calls, just intercepted by a stub library that forwards them over the local network. At that point you're making more work for yourself than debugging over SSH.
- tensor-fusion 6mo ago[dead]
- afavour 6mo agoChicken/egg. NVidia tooling is lacking surely in part because the hardware wasn’t usable on macOS until now. Now that it’s usable that might change.
- bigyabai 6mo agoNvidia tooling like CUDA has worked on AArch64 UNIX-certified OSes since June of 2020: https://download.nvidia.com/XFree86/Linux-aarch64/ https://download.nvidia.com/XFree86/Linux-aarch64/ The software stack has been ready for Apple Silicon for more than a half decade.
- frollogaston 6mo ago
- userbinator 6mo ago[flagged]
- mrits 6mo agoYou aren't restricted at a hardware level.
- ddtaylor 6mo agoApple has hardware level DRM in some of their products.
- llm_nerd 6mo agoSo you're just replying to the headline, not the actual article. Useful. Apple, just like Microsoft, has a driver signing process because drivers have basically system-wide access to a system. There is no evidence that nvidia has tried to get eGPU drivers signed for years, but now someone did and Apple signed it. So? And you could always, precisely as the article states in the very first paragraph, disable System Integrity Protection if you want to run drivers that aren't signed.
- u_fucking_dork 6mo ago[flagged]
- amelius 6mo agoYou only own the hardware if you can use it as advertised even after breaking all ties with the vendor. Otherwise you bought a service not a product.
- syntaxing 6mo agoFrom what I understand, only works with Tinygrad. Which is better than nothing but CUDA or Vulkan on pytorch isn’t going to work from this. [1] https://docs.tinygrad.org/tinygpu/ https://docs.tinygrad.org/tinygpu/
- vegabook 6mo ago[flagged]
- yjftsjthsd-h 6mo agoThey... do? Or rather, they built a system where they don't need to; macs happily run Linux on bare metal or VMs. (Whether Linux supports Apple hardware well is another matter)
- ece 6mo agoApple should update this page for ARM macs, now runs tinygrad on eGPUs: https://support.apple.com/en-us/102363 https://support.apple.com/en-us/102363
- lowbloodsugar 6mo agoCan I do prefill on the eGPU and the decode on the Mac?
- ajdegol 6mo agoI think that metal isn’t double precision; so that limits some serious physics simming; but if you’re doing that I guess you just rent a gpu somewhere. I would definitely be into this if adding an egpu was first class supported.
- nxobject 6mo agoIt'll be interesting to see whether this is price-competitive versus remoting into a cluster. Might be for smaller orgs/consultants.
- amelius 6mo agoThese tinyboxes are so expensive (starting at $12,000), why don't they just put a CPU inside and allow users to ssh into them?
- EagnaIonat 6mo ago> If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! I got an eGPU back in 2018 and could never get it to work. To the point that it soured me from doing it again. These days for heavy duty work I just offload to the cloud. This all feels like NVidia trying to be relevant versus ARM.
- embedding-shape 6mo ago> This all feels like NVidia trying to be relevant versus ARM. Except it's done by a third group, tinygrad, so it's more non-nvidia people wanting to use nvidia hardware one Apple hardware, than "nvidia trying to be relevant".
- EagnaIonat 6mo agoThanks for the correction. I guess my PTSD on trying to get this running before is bias'ing my response.
- ffsm8 6mo agoYeh, Nvidia couldn't give less of a fuck about consumers. And egpu is inherently only consumer targeted.
- bigyabai 6mo agoFWIW Nvidia already supports UNIX OSes and AArch64 with their drivers. CUDA and CUDNN could be working overnight if Apple signed the drivers.
- ErenalpCet 6mo agoyes good report
- direwolf20 6mo agoIsn't it sad that we've ended up in a situation where we are talking about "Apple approves" rather than "someone creates"? Fuck Apple.
- surcap526 6mo ago[dead]
- surcap526 6mo ago[dead]