5 ms·

Im tired of using AI in cloud services. I want user friendly locally owned AI hardware. Right now nothing is consumer friendly. I can’t get a packaged deal of

Im tired of using AI in cloud services. I want user friendly locally owned AI hardware. Right now nothing is consumer friendly. I can’t get a packaged deal of some locally running ChatGPT quality UI or voice command system in an all in one package. Like what Macs did for PCs I want the same for AI.

Hilift 1y ago
Oracle just announced they are spending $40 billion on GPU hardware. All cloud providers have an AI offering, and there are AI-specific cloud providers. I don't think retail is invited.
Disposal8433 1y ago
Your local computer is not powerful enough, and that's why you must welcome those brand new mainframes... I mean, "cloud services."
- pjmlp 1y ago
  It is funny how using a Web IDE, and a cloud shell, is such a déjà vu from when I used to do development on a common UNIX server shared by the whole team.
  - edoceo 1y ago
    Telnet from a Wyse terminal.
    - pjmlp 1y ago
      My first experience with such a setup was connecting to DG/UX, via the terminal application on Windows for Workgroups, or some thin client terminals in a mix of green or ambar phosphor, spread around the campus. The only time I used a Pascal compiler in ISO Pascal mode, it had the usual extensions inspired on UCSD, but we weren't allowed to use them on the assignments.
- ninetyninenine 1y ago
  My local computer is not powerful enough to run training but it can certainly run an LLM. How do I know? Many other people and I have already done it. Deepseek for example can be run locally but it’s not a user friendly setup. I want an Amazon echo agent running my home with a locally running LLM.
petesergeant 1y ago
Hoping the DGX Spark will deliver on this
- Gracana 1y ago
  It will not. 273GB/s memory bandwidth is not enough.
Workaccount2 1y ago
From the most unexpected place (but maybe expected if you believed they were paying attention) Maxsun is releasing a a 48GB dual Intel Arc Pro B60 GPU. It's expected to cost ~$1000. So for around $4k you should be able to build an 8 core 192GB local AI system, which would allow you to locally run some decent models. This also assumes the community builds an intel workflow, but given how greedy Nvidia is with vram, it seems poised to be a hit.
- zorgmonkey 1y ago
  The price of that system is unfortunately going to end up being a lot more than 4k, you'd need a CPU that has at least 64 lanes of PCIe. That's going to be either a Xeon W or a Threadripper CPU, with the motherboard RAM, etc you're probably looking at least another 2k. Also kind of a nitpick, but I'd call that 8 GPU system, each BMG-G21 die has 20 Xe2 cores. Also even though it would be 4 PCIe cards it is probably best to think of it as 8 GPUs (that's how it will show up in stuff like pytorch), especially because their is no high-speed interconnect between the GPU dies colocated on the card. Also if you're going to do this make sure you get a motherboard with good PCIe bifurcation support.
ata_aman 1y ago
I made something[0] last year to have something very consumer friendly. Unbox->connect->run. First iteration is purely to test out the concept and is pretty low power, currently working on a GPU version for bigger models and launching Q4 this year. [0] https://persys.ai https://persys.ai

Hilift 1y ago

Oracle just announced they are spending $40 billion on GPU hardware. All cloud providers have an AI offering, and there are AI-specific cloud providers. I don't think retail is invited.

Disposal8433 1y ago

Your local computer is not powerful enough, and that's why you must welcome those brand new mainframes... I mean, "cloud services."

pjmlp 1y ago
It is funny how using a Web IDE, and a cloud shell, is such a déjà vu from when I used to do development on a common UNIX server shared by the whole team.
- edoceo 1y ago
  Telnet from a Wyse terminal.
  - pjmlp 1y ago
    My first experience with such a setup was connecting to DG/UX, via the terminal application on Windows for Workgroups, or some thin client terminals in a mix of green or ambar phosphor, spread around the campus. The only time I used a Pascal compiler in ISO Pascal mode, it had the usual extensions inspired on UCSD, but we weren't allowed to use them on the assignments.
ninetyninenine 1y ago
My local computer is not powerful enough to run training but it can certainly run an LLM. How do I know? Many other people and I have already done it. Deepseek for example can be run locally but it’s not a user friendly setup. I want an Amazon echo agent running my home with a locally running LLM.

petesergeant 1y ago

Hoping the DGX Spark will deliver on this

Gracana 1y ago
It will not. 273GB/s memory bandwidth is not enough.

Workaccount2 1y ago

From the most unexpected place (but maybe expected if you believed they were paying attention) Maxsun is releasing a a 48GB dual Intel Arc Pro B60 GPU. It's expected to cost ~$1000. So for around $4k you should be able to build an 8 core 192GB local AI system, which would allow you to locally run some decent models. This also assumes the community builds an intel workflow, but given how greedy Nvidia is with vram, it seems poised to be a hit.

zorgmonkey 1y ago
The price of that system is unfortunately going to end up being a lot more than 4k, you'd need a CPU that has at least 64 lanes of PCIe. That's going to be either a Xeon W or a Threadripper CPU, with the motherboard RAM, etc you're probably looking at least another 2k. Also kind of a nitpick, but I'd call that 8 GPU system, each BMG-G21 die has 20 Xe2 cores. Also even though it would be 4 PCIe cards it is probably best to think of it as 8 GPUs (that's how it will show up in stuff like pytorch), especially because their is no high-speed interconnect between the GPU dies colocated on the card. Also if you're going to do this make sure you get a motherboard with good PCIe bifurcation support.

ata_aman 1y ago

I made something[0] last year to have something very consumer friendly. Unbox->connect->run. First iteration is purely to test out the concept and is pretty low power, currently working on a GPU version for bigger models and launching Q4 this year. [0] https://persys.ai https://persys.ai