3 ms·
This feels solvable to me. I wonder if you could use fine tuning against LLaMA to teach it to do this better? GPT-3 etc can only do this because they had a LOT
by simonw 4y ago
This feels solvable to me. I wonder if you could use fine tuning against LLaMA to teach it to do this better?
GPT-3 etc can only do this because they had a LOT of code included in their training sets.
The LLaMA paper says Github was 4.5% of the training corpus, so maybe it does have that stuff baked in and just needs extra tuning or different prompts to tap into that knowledge.
- avereveard 4y agoI have done it trough stages, so first stages emits in natural language in the format of "context: ... and question: ...." and then the second stage collect it as json, but then wait time doubles.