4 ms·
are we talking about training or inference for local LLM here? it's hard to do any meaningful training on the edge unless we all carry a heavy gaming pc, even t
by synergy20 4y ago
are we talking about training or inference for local LLM here? it's hard to do any meaningful training on the edge unless we all carry a heavy gaming pc, even that, the training quality will be subpar?
- qumpis 4y agoInference, even fine-tuning a few layers would be difficult since one needs to use non-quantized model, I'd imagine
- ctoth 4y agoCheckout LoRA and Alpaca LoRA and the whole huge group of people who have already figured this out. I think there was another breakthrough (yesterday?) which is a further adaption of LoRA to touch even less parameters at runtime.
- homarp 4y agoDyLoRA https://news.ycombinator.com/item?id=35514228 https://news.ycombinator.com/item?id=35514228 is what you meant
- whimsicalism 4y agoDisagree that this is a major breakthrough :) but it is probably a marginal improvement.
- whimsicalism 4y agoThey're not training on edge :)