4 ms·
Something like this is nice, where instead of having 1 model with X active experts, you have 10 different models, all small and dense, trained on specific infor
by unrvl22 3mo ago
Something like this is nice, where instead of having 1 model with X active experts, you have 10 different models, all small and dense, trained on specific information. and loaded on 10 different servers, with one router.
- Abishek_Muthian 3mo agoTrue, I've been telling that coding LLMs have been detrimental to local AI scene. What happened to those nifty ML projects which can run on raspberry pi zero.
- sig_kill 3mo agoThis is actually how I develop and use the mesh at home. Rather than splitting models, I aggregate disparate compute behind one endpoint, without having separate inference providers on each host and a gateway like LiteLLM