3 ms·
Well... I wanted a dedicated AI/ML box here at home, with enough horsepower to do some interesting AI stuff (but without breaking the bank totally). So I built
by mindcrime 3y ago
Well... I wanted a dedicated AI/ML box here at home, with enough horsepower to do some interesting AI stuff (but without breaking the bank totally). So I built myself a new PC from parts for the first time in around 20 years. That was ... interesting. Building a PC is - in many ways - not that different than it was in 2005. But it's also changed in many ways. So I spent a lot of time researching and picking out parts, and then building the system. Which led me to things like spending hours watching youtube videos on "What's the best thermal paste" and "what's the best CPU cooler" and "are thermal pads better or worse than thermal paste" and "Do you need one of these CPU gasket things if you're running socket AM5" and so on.
I just got the box finished up over the weekend and it's working really nicely so far. I went with an MSI MEG x670e "Godlike" motherboard, AMD Ryzen 9 7950X CPU, MSI Gaming Radeon RX 7900 XTX GPU, and 64GB of DDR5 RAM. The whole thing is running Ubuntu 22.04 (I was going to use Alma Linux, but the Alma installer wouldn't even start), ROCm 6.1.0 and Ollama. So far I've mostly been working with the LLM stuff using Ollama and the llama3 model.
Now I've started using Spring AI to interface with the Ollama API. Next steps: figure out "function calling" with Ollama (which doesn't seem to be supported by Spring AI yet, boo) and some "agentic workflows" and multi-agent stuff...