2 ms·

Linux for me. Mac is way too expensive especially if you want lots of RAM which you need for most AI purposes. Windows is an ad-ridden spyware fest these days a

Linux for me. Mac is way too expensive especially if you want lots of RAM which you need for most AI purposes. Windows is an ad-ridden spyware fest these days and you don't have control over updates and reboots. So both Mac and Windows are non-starters for me. I played a bit with things on windows but I'd never use it for something serious. Microsoft invests a lot into their AI tooling now but I don't want to be beholden to them. I have to deal with them at work but I don't trust them. I built a Linux server for AI LLM inference with a Radeon Pro card and it was really good value for money. It took some doing to get it working especially with the lesser AMD compatibility but I saved a ton of money not buying Nvidia and it performs excellently now. Windows probably would have been easier to set up but I consider every configuration problem a learning experience and my freedom is worth more than my time. The reason the Mac is so popular is that it's the only way to get a GPU with direct access to tons of memory (up to 192GB) and it's cheaper than Nvidia dedicated AI options. But both have astronomical costs not justifiable to a hobbyist like me.

shivc 2y ago
That's quite an insight. I'm gravitating more towards linux especially after this copilot+ shit microsoft pulled out from the magic bag of tricks. I noticed issues with nvidia too on linux however arch seems to run things pretty smooth for me compared to other debian based distros, so probably going to go with that. I've been running arch in an nvme with an enclosure through usb c and tbh it is amazing but now considering just removing the windows drive and sticking with linux as the main driver while keeping windows in the enclosure since I do game time to time. As for mac, have had an amazing overall experience with m1 but then i got the base model since i didn't have a few extra kidneys in me to sell to afford the max lol but yes, seems like linux takes it home!
- wkat4242 2y ago
  Yeah it just took some doing getting ROCm to actually work. Installing it was easy, but getting it to work not quite so. In the end it works fine though. Ollama supports it well, but be aware that many other toolkits don't because it simply won't support CUDA. The card was really excellent value for money though. 300 euro for a Radeon Pro VII with 16TB of HBM2 memory. I run Ubuntu as it was better supported by the drivers. Not a big fan of ubuntu but it's just a server, not my daily driver.
  - shivc 2y ago
    I've had to go for a dual boot for most of my stuff as I like to work linux but tons of tools that work in windows better that I need to use for work. AMD GPUs are damn good value, in fact even AMD processors tend to pack quite an amazing punch compared to what we'd get with intel in the same price brackets. Linux all the way!
- wkat4242 2y ago
  By the way yeah I know what you mean about Microsoft. I work heavily in AI and our company is heavily invested in Microsoft tooling. But personally I like to build everything myself. I don't like to be locked into the world according to microsoft. Especially because these guys are so annoying to deal with. They're only a supplier but they're talking to us like they're our boss. At work I put up with it but things I do for my personal benefit (like learning) I do on my own. All my coworkers were pumped up watching Microsoft Build this week, but I'm playing with ollama and making tools to make it talk to all my own systems. To me that's a much better way of really understanding how things work, to touch the nuts & bolts myself, not to outsource the difficult parts. I also hate classroom trainings (or online), I just don't have patience for that shit.

shivc 2y ago

That's quite an insight. I'm gravitating more towards linux especially after this copilot+ shit microsoft pulled out from the magic bag of tricks. I noticed issues with nvidia too on linux however arch seems to run things pretty smooth for me compared to other debian based distros, so probably going to go with that. I've been running arch in an nvme with an enclosure through usb c and tbh it is amazing but now considering just removing the windows drive and sticking with linux as the main driver while keeping windows in the enclosure since I do game time to time. As for mac, have had an amazing overall experience with m1 but then i got the base model since i didn't have a few extra kidneys in me to sell to afford the max lol but yes, seems like linux takes it home!

wkat4242 2y ago
Yeah it just took some doing getting ROCm to actually work. Installing it was easy, but getting it to work not quite so. In the end it works fine though. Ollama supports it well, but be aware that many other toolkits don't because it simply won't support CUDA. The card was really excellent value for money though. 300 euro for a Radeon Pro VII with 16TB of HBM2 memory. I run Ubuntu as it was better supported by the drivers. Not a big fan of ubuntu but it's just a server, not my daily driver.
- shivc 2y ago
  I've had to go for a dual boot for most of my stuff as I like to work linux but tons of tools that work in windows better that I need to use for work. AMD GPUs are damn good value, in fact even AMD processors tend to pack quite an amazing punch compared to what we'd get with intel in the same price brackets. Linux all the way!
wkat4242 2y ago
By the way yeah I know what you mean about Microsoft. I work heavily in AI and our company is heavily invested in Microsoft tooling. But personally I like to build everything myself. I don't like to be locked into the world according to microsoft. Especially because these guys are so annoying to deal with. They're only a supplier but they're talking to us like they're our boss. At work I put up with it but things I do for my personal benefit (like learning) I do on my own. All my coworkers were pumped up watching Microsoft Build this week, but I'm playing with ollama and making tools to make it talk to all my own systems. To me that's a much better way of really understanding how things work, to touch the nuts & bolts myself, not to outsource the difficult parts. I also hate classroom trainings (or online), I just don't have patience for that shit.