4 ms·

I don't understand why folks are buying Mac Minis specifically for this? Why not repurpose an old existing computer? Run Linux? What am I missing?

biztos 8mo ago
If you're running local models, Apple Silicon's shared memory architecture makes them much better at it than other similarly-specced platforms. If you want your "skills" to include sending iMessage (quite important in the USA), then you need a Mac of some kind. If you don't care about iMessage and you're just doing API calls for the inference, then it's good old Mass Abundance. Nice excuse to get that cool little Mini you've been wanting.
correct_horse 8mo ago
Mac minis are particularly suited to running AI models because they can have a pretty good quantity of RAM (64GB) assigned to the GPU at a reasonable price compared to Nvidia offerings. Mac minis have unified memory which means it can be split between CPU and GPU in a configurable way. I think apple didn’t price mac minis with AI stuff in mind, so they end up being good value.
- sleight42 8mo ago
  Sure but the GPUs are fairly anemic, right? I get that they have more Gpu-addressable memory from the shared pool. I have a 10900K with 65GB RAN and a 3090 24GB VRAM lying around gathering dust. 24GB isn't as much as a Mac but my cores run a whole lot faster. I may be able to run a 34B 4bit quantized model in that. Granted, the mofo will eat a lot of power.
denkmoon 8mo ago
Where do you get the AI acceleration? Apple Silicon chips are decent AI perf for the price afaiu
- rocmcd 8mo ago
  There's no need for (local) AI acceleration if you are leveraging a remote LLM (Claude, ChatGPT, etc). The vast, vast majority of users are most likely just making API calls to a remote service. No need for specialized or beefy hardware.
caminante 8mo ago
Hype and confusion. OpenClaw is hyped for running local/private LLMs and controlling your data, but these people don't realize the difference between (1) running local open source LLMs (2) and API calls to cloud LLMs. The vast majority will do #2. To your point, a Raspberry Pi is sufficient. For the former, you still need a lot of RAM (+32GB for larger models) so most minis are underpowered despite having unified memory and higher efficiency.
- h14h 8mo ago
  Yup. Been building my own "Claw" in Go using cloud LLMs and it's running very happily on a $6/mo VPS with 1 vCPU and 1GB of RAM.
flutas 8mo ago
While others will point to hardware or local LLMs or such IMO the biggest reason... Because it's the easiest way to give "claw" iMessage access and that's the primary communication channel for a lot of the claw users I've seen.

biztos 8mo ago

If you're running local models, Apple Silicon's shared memory architecture makes them much better at it than other similarly-specced platforms. If you want your "skills" to include sending iMessage (quite important in the USA), then you need a Mac of some kind. If you don't care about iMessage and you're just doing API calls for the inference, then it's good old Mass Abundance. Nice excuse to get that cool little Mini you've been wanting.

correct_horse 8mo ago

Mac minis are particularly suited to running AI models because they can have a pretty good quantity of RAM (64GB) assigned to the GPU at a reasonable price compared to Nvidia offerings. Mac minis have unified memory which means it can be split between CPU and GPU in a configurable way. I think apple didn’t price mac minis with AI stuff in mind, so they end up being good value.

sleight42 8mo ago
Sure but the GPUs are fairly anemic, right? I get that they have more Gpu-addressable memory from the shared pool. I have a 10900K with 65GB RAN and a 3090 24GB VRAM lying around gathering dust. 24GB isn't as much as a Mac but my cores run a whole lot faster. I may be able to run a 34B 4bit quantized model in that. Granted, the mofo will eat a lot of power.

denkmoon 8mo ago

Where do you get the AI acceleration? Apple Silicon chips are decent AI perf for the price afaiu

rocmcd 8mo ago
There's no need for (local) AI acceleration if you are leveraging a remote LLM (Claude, ChatGPT, etc). The vast, vast majority of users are most likely just making API calls to a remote service. No need for specialized or beefy hardware.

caminante 8mo ago

Hype and confusion. OpenClaw is hyped for running local/private LLMs and controlling your data, but these people don't realize the difference between (1) running local open source LLMs (2) and API calls to cloud LLMs. The vast majority will do #2. To your point, a Raspberry Pi is sufficient. For the former, you still need a lot of RAM (+32GB for larger models) so most minis are underpowered despite having unified memory and higher efficiency.

h14h 8mo ago
Yup. Been building my own "Claw" in Go using cloud LLMs and it's running very happily on a $6/mo VPS with 1 vCPU and 1GB of RAM.

flutas 8mo ago

While others will point to hardware or local LLMs or such IMO the biggest reason... Because it's the easiest way to give "claw" iMessage access and that's the primary communication channel for a lot of the claw users I've seen.