4 ms·

> ...making the perfect hardware for home inference machines. I really don't get why anybody would want that. What's the use case there? If someone doesn't ca

by utopiah 7mo ago

> ...making the perfect hardware for home inference machines. I really don't get why anybody would want that. What's the use case there? If someone doesn't care about privacy, they can use for-profit services because they are basically losing money, trying to corner the market. If they care about privacy, they can rent cloud instances in order to setup, run, close and it will be both cheaper, faster (if they can afford it) but also with no upfront cost per project. This can be done with a lot of scaffolding, e.g. Mistral, HuggingFace, or not, e.g. AWS/Azure/GoogleCloud, etc. The point being that you do NOT purchase the GPU or even dedicated hardware, e.g. Google TPU, but rather rent for what you actually need and when the next gen is up, you're not stuck with "old" gen. So... what use case if left, somebody who is both technical, very privacy conscious AND want to do so offline despite have 5G or satellite connectivity pretty much anywhere? I honestly don't get who that's for (and I did try a dozens of local models, so I'm actually curious). PS: FWIW https://pricepertoken.com https://pricepertoken.com might help but not sure it shows the infrastructure each rely on to compare. If you have a better link please share back.

pwython 7mo ago
Genuine question: If I were to fine-tune a model with 10 years of business data in a competitive space, would you feel safe with cloud training?
- yomismoaqui 7mo ago
  If you already have those 10 years of bussiness data on Microsoft or Google services or their respective clouds, are you feeling safe?
- justinhj 7mo ago
  People store that data in databases in the same data centre so it's really the same level of trust needed that your provider adheres to the no training on your data. Trust and lawyers.
- utopiah 7mo ago
  I'm not a lawyer but technically most if not all cloud providers, specific to AI ("neo-cloud") or not, to provide Customer-managed encryption keys (CMEK) as someone else pointed out. That being said if I were to be in such a situation, and if somehow the guarantees wouldn't be enough then I'd definitely expect to have the budget to build my own data center with GB300 or TPUs. I can't imagine that running it on a Mac Studio.
BeetleB 7mo ago
> If they care about privacy, they can rent cloud instances in order to setup, run, close and it will be both cheaper, faster (if they can afford it) but also with no upfront cost per project. This can be done with a lot of scaffolding, e.g. Mistral, HuggingFace, or not, e.g. AWS/Azure/GoogleCloud, etc. I'm a somewhat tech heavy guy (compiles my own kernel, uses online hosting, etc). Reading your comment doesn't sound appealing at all. I do almost no cloud stuff. I don't know which provider to choose. I have to compare costs. How can I trust they won't peek at my data (no, a Privacy Policy is not enough - I'd need encryption with only me having the key). What do I do if they suddenly jack up the rates or go out of business? I suddenly need a backup strategy as well. And repeat the whole painful loop. I'll lose a lot more time figuring this out than with a Mac Studio. I'll probably lose money too. I'll rent from one provider, get stuck, and having a busy life, sit on it a month or two before I find a fix (paying money for nothing). At least if I use the Mac Studio as my primary machine, I don't have to worry about money going to waste because I'm actually utilizing it. And chances are, a lot of the data I'll use it with (e.g. mail) is sitting on the same machine anyway. Getting something on the cloud to work with it is yet-another-pain.
- eitally 7mo ago
  To your second issue/question, all the cloud provide CMEK services/features (for many years now).
- utopiah 7mo ago
  > suddenly jack up the rates or go out of business? There is basically no lock-in, you don't even "move" your image, your data is basically some "context" or a history of prompts which probably fits in a floppy disk (not even being sarcastic) so if you know the basic about containerization (Docker, podman, etc) which most likely the cloud provider even takes care of, then it takes literally minutes to switch from one to another. It's really not more complex that setting up a PHP server, the only difference is the hardware you run on and that's basically a dropdown button on a Web interface (if you don't want to have scripts for that too) then selecting the right image (basically NVIDIA support). Consequently even if that were to happen (which I have NEVER seen! at worst it's like 15% increase after years) then it would actually not matter to you. It's also very unlikely to happen based of the investment poured into the "industry". Basically everybody is trying to get "you" as a customer to rely on their stack. ... but OK, let's imagine that's not appealing to you, have you not done the comparison of what a Mac Studio (or whatever hardware) could actually buy otherwise?
jedberg 7mo ago
I think the main use case is home automation. You don't want details of your home setup leaking out.
- utopiah 7mo ago
  I've been running home automation 24/7 for few years now, using Mozilla WebThings then HomeAssistant. I indeed selected my hardware to rely on proper standards, e.g. ZigBee, so that I wouldn't have to use any proprietary app. For me it's mostly about resilience, e.g work offline, than privacy but of course I also appreciate that benefit. ... that being said, and I did read https://www.home-assistant.io/blog/2025/09/11/ai-in-home-assistant/ https://www.home-assistant.io/blog/2025/09/11/ai-in-home-ass... or even have use a local modal for STT but IMHO it's not good, definitely not worth buying a $4k device for. It doesn't generate good rules, it gets some commands but it's quite basic. So I understand the concept but so far I haven't seen anything good enough to warrant such a purchase for such a very very specific niche. I can imagine literally a couple of persons interested in this but I'd bet they'd also have a GPU laying around that could be available for that too. TL;DR: technically not impossible, I doubt it's a popular use case.

by utopiah 7mo ago

pwython 7mo ago

Genuine question: If I were to fine-tune a model with 10 years of business data in a competitive space, would you feel safe with cloud training?

yomismoaqui 7mo ago
If you already have those 10 years of bussiness data on Microsoft or Google services or their respective clouds, are you feeling safe?
justinhj 7mo ago
People store that data in databases in the same data centre so it's really the same level of trust needed that your provider adheres to the no training on your data. Trust and lawyers.
utopiah 7mo ago
I'm not a lawyer but technically most if not all cloud providers, specific to AI ("neo-cloud") or not, to provide Customer-managed encryption keys (CMEK) as someone else pointed out. That being said if I were to be in such a situation, and if somehow the guarantees wouldn't be enough then I'd definitely expect to have the budget to build my own data center with GB300 or TPUs. I can't imagine that running it on a Mac Studio.

BeetleB 7mo ago

> If they care about privacy, they can rent cloud instances in order to setup, run, close and it will be both cheaper, faster (if they can afford it) but also with no upfront cost per project. This can be done with a lot of scaffolding, e.g. Mistral, HuggingFace, or not, e.g. AWS/Azure/GoogleCloud, etc. I'm a somewhat tech heavy guy (compiles my own kernel, uses online hosting, etc). Reading your comment doesn't sound appealing at all. I do almost no cloud stuff. I don't know which provider to choose. I have to compare costs. How can I trust they won't peek at my data (no, a Privacy Policy is not enough - I'd need encryption with only me having the key). What do I do if they suddenly jack up the rates or go out of business? I suddenly need a backup strategy as well. And repeat the whole painful loop. I'll lose a lot more time figuring this out than with a Mac Studio. I'll probably lose money too. I'll rent from one provider, get stuck, and having a busy life, sit on it a month or two before I find a fix (paying money for nothing). At least if I use the Mac Studio as my primary machine, I don't have to worry about money going to waste because I'm actually utilizing it. And chances are, a lot of the data I'll use it with (e.g. mail) is sitting on the same machine anyway. Getting something on the cloud to work with it is yet-another-pain.

eitally 7mo ago
To your second issue/question, all the cloud provide CMEK services/features (for many years now).
utopiah 7mo ago
> suddenly jack up the rates or go out of business? There is basically no lock-in, you don't even "move" your image, your data is basically some "context" or a history of prompts which probably fits in a floppy disk (not even being sarcastic) so if you know the basic about containerization (Docker, podman, etc) which most likely the cloud provider even takes care of, then it takes literally minutes to switch from one to another. It's really not more complex that setting up a PHP server, the only difference is the hardware you run on and that's basically a dropdown button on a Web interface (if you don't want to have scripts for that too) then selecting the right image (basically NVIDIA support). Consequently even if that were to happen (which I have NEVER seen! at worst it's like 15% increase after years) then it would actually not matter to you. It's also very unlikely to happen based of the investment poured into the "industry". Basically everybody is trying to get "you" as a customer to rely on their stack. ... but OK, let's imagine that's not appealing to you, have you not done the comparison of what a Mac Studio (or whatever hardware) could actually buy otherwise?

jedberg 7mo ago

I think the main use case is home automation. You don't want details of your home setup leaking out.

utopiah 7mo ago
I've been running home automation 24/7 for few years now, using Mozilla WebThings then HomeAssistant. I indeed selected my hardware to rely on proper standards, e.g. ZigBee, so that I wouldn't have to use any proprietary app. For me it's mostly about resilience, e.g work offline, than privacy but of course I also appreciate that benefit. ... that being said, and I did read https://www.home-assistant.io/blog/2025/09/11/ai-in-home-assistant/ https://www.home-assistant.io/blog/2025/09/11/ai-in-home-ass... or even have use a local modal for STT but IMHO it's not good, definitely not worth buying a $4k device for. It doesn't generate good rules, it gets some commands but it's quite basic. So I understand the concept but so far I haven't seen anything good enough to warrant such a purchase for such a very very specific niche. I can imagine literally a couple of persons interested in this but I'd bet they'd also have a GPU laying around that could be available for that too. TL;DR: technically not impossible, I doubt it's a popular use case.