3 ms·
LLMs can do that without any examples (zero shot) or with one or a few demonstrations in the prompt, if you can describe the task in the limited context window.
by visarga 3y ago
LLMs can do that without any examples (zero shot) or with one or a few demonstrations in the prompt, if you can describe the task in the limited context window.
If you want for example to train the model to learn to use a very large API, or access the knowledge in a whole book, it might need fine-tuning.
- nico 3y agoCould I just train a very small LLM with an English dictionary + Python + large API documentation + large Python code base? Then do some chat fine tuning (like what HF did with StarCoder to get ChatCoder) And get a lightweight LLM that knows the docs and code for the thing I need it for After that, maybe incrementally fine tune the model as part of your CI/CD process