3 ms·
I wonder if it is better to have a machine somewhere running a model for you maybe shared with a few others. I could probably justify a M6 Mac Studio with hopef
by andy_ppp 4mo ago
I wonder if it is better to have a machine somewhere running a model for you maybe shared with a few others. I could probably justify a M6 Mac Studio with hopefully 256gb RAM and have a few people all with access to one agreed upon model. I think maybe laptops are too warm and clunky for this.
- wgd 4mo agoThe problem is that the moment you introduce shared remote hardware there's a slippery slope leading right back down to "just pay an inference host for model tokens". If you're transmitting your prompts over the internet to a trusted host you might as well just let that host be DeepInfra or together.ai or one of the many other providers already in that business.
- andy_ppp 4mo agoI dunno, I probably need the web to be able to do work so why does it matter - taking the simple case - of running just myself on a Mac Studio at home or cooking my self on the go I'd probably rather have a cheaper laptop and dedicated hardware. I think for many this is about having control over the model and not about farming things out to a SAAS... what does the saying say opinions are like again.