3 ms·

I keep thinking there should be a way to train a copilot against just one set of code libraries. I know LLMs require training against a lot of text to get their

I keep thinking there should be a way to train a copilot against just one set of code libraries. I know LLMs require training against a lot of text to get their smarts, but is there a way to set this up so a model can be created for a specific library by anyone, so it could provide open source support via a transformer + model? Maybe this would be a better approach than a jack of all trades, master of none.

nl 3y ago
Yes, this is what fine tuning is for. It's pretty obvious that lots of people will want to take a strong code completion model, then fine tune it on their docs + libraries and then make it available inside their docs/discord/slack as a support thing.
- davidy123 3y ago
  I guess as soon as a kit is available for this purpose that doesn't require advanced knowledge (say, `aisupport4 my-repo/`) and runs on mainstream-ish hardware and doesn't require a centralized service (even running in the browser via eg transformers.js), things will change considerably.
  - nl 3y ago
    As someone who is very interested in decentralized services (as in my day job involves decentralized databases and I'm actively working on WebGPU support for training) I'd say that the browser-based vision is a fair way off. The software ecosystem is pretty immature, and there are numerous things that need to change before the core technologies are good enough to fine tune competitive LLMs. I do think fine tuning moderate sized LLMs on your own (pretty expensive) hardware using consumer GPUs maybe possible this year. Unfortunately all the evidence is that training (as opposed to inference) requires high-precision, and hence high memory. This is something that consumer GPUs for the most part lack. New techniques are likely to be required (eg better sharing of training on low memory GPUs) but it's hard to predict how they will develop.

nl 3y ago

Yes, this is what fine tuning is for. It's pretty obvious that lots of people will want to take a strong code completion model, then fine tune it on their docs + libraries and then make it available inside their docs/discord/slack as a support thing.

davidy123 3y ago
I guess as soon as a kit is available for this purpose that doesn't require advanced knowledge (say, `aisupport4 my-repo/`) and runs on mainstream-ish hardware and doesn't require a centralized service (even running in the browser via eg transformers.js), things will change considerably.
- nl 3y ago
  As someone who is very interested in decentralized services (as in my day job involves decentralized databases and I'm actively working on WebGPU support for training) I'd say that the browser-based vision is a fair way off. The software ecosystem is pretty immature, and there are numerous things that need to change before the core technologies are good enough to fine tune competitive LLMs. I do think fine tuning moderate sized LLMs on your own (pretty expensive) hardware using consumer GPUs maybe possible this year. Unfortunately all the evidence is that training (as opposed to inference) requires high-precision, and hence high memory. This is something that consumer GPUs for the most part lack. New techniques are likely to be required (eg better sharing of training on low memory GPUs) but it's hard to predict how they will develop.