3 ms·
Good value but a 12GB card isn't going to let you do too much given the low quality of small models. Curious what "home AI" use cases small models are being use
by mythz 1y ago
Good value but a 12GB card isn't going to let you do too much given the low quality of small models. Curious what "home AI" use cases small models are being used for?
It would be nice to see a best value home AI setups under different budgets or RAM tiers, e.g. best value configuration for 128 GPU VRAM, etc.
My 48GB GPU VRAM "Home AI Server" cost ~$3100 from all parts on eBay running 3x A4000's in a Supermicro 128GB RAM, 32/64 core Xeon 1U rack server. Nothing amazing but wanted the most GPU VRAM before paying the premium Nvidia tax on their larger GPUs.
This works well for Ollama/llama-server which can make use of all GPU VRAM unfortunately ComfyUI can't make use of all GPU VRAM to run larger models, so on the lookout for a lot more RAM in my next GPU Server.
Really hoping Intel can deliver with its upcoming Arc Pro B60 Dual GPU for a great value 48GB option which can be run 4x in an affordable 192GB VRAM workstation [1]. If it runs Ollama and ComfyUI efficiently I'm sold.
[1] https://www.servethehome.com/maxsun-intel-arc-pro-b60-dual-gpu-48gb-at-computex-2025/ https://www.servethehome.com/maxsun-intel-arc-pro-b60-dual-g...
- itake 1y agoMy home AI machine does image classification.
- mythz 1y agoUsing just an Ollama VL Model (gemma3/mistral-small3.1/qwen2.5vl) or a specific library?
- itake 1y agoMy home server detects NSFW images in user generated content on my side project. source code: https://github.com/KevinColemanInc/NSFW-FLASK https://github.com/KevinColemanInc/NSFW-FLASK
- mythz 1y agoCool, I've tried a few but settled on using EraX NSFW to do the same.
- itake 1y agojust wondering, why that one? it looks small, which is probably doesn't require a gpu. The dataset seems to be images of high production value (e.g. limited races, staged poses, etc). If I have time, I will compare it with Bumble's model, but I think the images I'm trying to identify are closer to Bumble's training set.
- naavis 1y agoWhat kind of image classification do you do at home?
- itake 1y agoMy side project accepts and publishes user generated content. To stay compliant with regulations, I use ML to remove adult content: https://github.com/KevinColemanInc/NSFW-FLASK https://github.com/KevinColemanInc/NSFW-FLASK
- deleted 1y ago[deleted]
- jononor 1y agoAgreed, 12 GB does not seem useful. For coding LLM, it seems 128 GB is needed to be even close to the frontier models. For generative image processing (not video), it looks like one can get started with 16GB.
- msgodel 1y agoIt's really not going to let you train much which IMO is the only reason I'd personally bother with a big GPU. Gradients get huge and everything does them with single/half precision floating point.
- rwyinuse 1y agoI use a Proxmox server with RTX 3060 to generate paintings (I have a couple of old jailbroken Amazon Kindle's attached to walls for that purpose), and to run ollama, which is connected to Home Assistant & their voice preview device, allowing me to talk with LLM without transmitting anything to cloud services. Admittedly with that amount of VRAM the models I can run are fairly useless for stuff like controlling lights via Home Assistant, occasionally does what I tell it to do but usually not. It is pretty okay for telling me information, like temperature or value of some sensors I have connected to HA. For generating AI paintings it's enough. My server also hosts tons of virtual machines, docker containers and is used for remote gameplay, so the AI thing is just an extra.
- garyfirestorm 1y agoWhy do you say that? You can easily finetune 8B parameter model for function calling.
- rwyinuse 1y agoThat's good to know. The model I was using supported function calling, but seemed to get the calls often wrong. Perhaps I should try a more fine-tuned model for the purpose.