3 ms·
Inference, even fine-tuning a few layers would be difficult since one needs to use non-quantized model, I'd imagine
by qumpis 4y ago
Inference, even fine-tuning a few layers would be difficult since one needs to use non-quantized model, I'd imagine
- ctoth 4y agoCheckout LoRA and Alpaca LoRA and the whole huge group of people who have already figured this out. I think there was another breakthrough (yesterday?) which is a further adaption of LoRA to touch even less parameters at runtime.
- homarp 4y agoDyLoRA https://news.ycombinator.com/item?id=35514228 https://news.ycombinator.com/item?id=35514228 is what you meant
- whimsicalism 4y agoDisagree that this is a major breakthrough :) but it is probably a marginal improvement.
- whimsicalism 4y agoThey're not training on edge :)