3 ms·
- in context learning (simply put it into context) - finetuning, as in restarting the models checkpoint and relearning it on the previous + new data - adding
by eurekin 3y ago
- in context learning (simply put it into context)
- finetuning, as in restarting the models checkpoint and relearning it on the previous + new data
- adding extra neurons (e.g. LoRA adapters) at certain places and restarting learning
Oh, in classic machine learning there's also the "bagging/boosting classifiers" option, but I have no knowledge if that can be applied to a ANN.
- simonw 3y agoHave you seen LoRA adapters actually work for this kind of thing? The leaked Google "We have no moat" memo was very excited about LoRA style techniques, but it's not clear to me that it's been proven as a technique yet.
- eurekin 3y agoNo, I have tried about 3 times and all failed. Altough it might be that I'm simply doing it wrong. There are people (can point at a discord server) claiming it works for them and that they even sell finetuned models to business clients. EDIT: I found one of the articles I tried to follow: https://www.mlexpert.io/prompt-engineering/chatbot-with-local-llm-using-langchain https://www.mlexpert.io/prompt-engineering/chatbot-with-loca... EDIT2: Ignore above. This seemed much more promising: https://www.youtube.com/watch?v=pnwVz64jNvw https://www.youtube.com/watch?v=pnwVz64jNvw . Author provides consulting services and seemed very nice and approachable