3 ms·
interesting, I hadn't thought about the problem where they don't follow the instruction "only output the JSON, do not add explanations or other text" I haven't
by verdverm 3y ago
interesting, I hadn't thought about the problem where they don't follow the instruction "only output the JSON, do not add explanations or other text"
I haven't pushed on codellama2 much yet, but my initial experiments, it did not really output anything extra, and my prompt became a one-liner compared to the really long instructions I had to give chatgpt for controlling output. Shows how far you can get with a purpose trained model
fine-tuning is important to getting more consistent output, none of the smaller models (open-source sized) are going to get there with just few-shot. Sounds like the grammar logit influencer is a low-cost/effort way to constrain output without the fine-tuning cycle. I can imagine they might be better together, but my hunch is that fine-tuning will still dominate the improvements and consistency. If you don't have the training data, that is a very good reason to use this technique too
- simonw 3y agoMy favourite joke about LLMs not following formatting instructions is this from Riley Goodside: https://twitter.com/goodside/status/1657396491676164096 https://twitter.com/goodside/status/1657396491676164096