3 ms·
Because running it locally really means running it on a cloud server that you own and is called by other server that you own. This gives you the ability to make
by sgt101 3y ago
Because running it locally really means running it on a cloud server that you own and is called by other server that you own. This gives you the ability to make the interfaces lightweight and most importantly to not pay premiums to model servers.
- jameshart 3y agoNo, running it locally means running it on my laptop. My Mac M2 is quite capable of running stable diffusion XL models and 30M parameter. LLMs under llama.cpp. What I don’t like is the trend towards the way to do that being to open up network listeners with no authentication on them.
- teaearlgraycold 3y agoBind to localhost then
- karolist 3y agolmstudio my friend
- sgt101 3y ago>What I don’t like is the trend towards the way to do that being to open up network listeners with no authentication on them. Yeah - but don't do that. The thing about small models that can run on commodity hardware is that it breaks the business model of OpenAI and co. They hope that they can run a service that charges a fortune but provides functionality that can't be duplicated. This gives them a moat and a huge revenue engine. Quantized models and student models (trained from the big models outputs) show that the moat is likely to be transitory or partial at best. We can run Mistral 7B at about 1/300th of the cost of a call to GPT4. That makes a whole load of applications viable, but it also torpedoes the monopoly pricing model that they are hoping for. All we need to do now is to stop people training on stolen data.