4 ms·
I'm curious: did you have to tailor your dataset around instruction-following/reasoning capabilities as well? No conflict of interest myself – I'm interested in
by nxobject 1y ago
I'm curious: did you have to tailor your dataset around instruction-following/reasoning capabilities as well? No conflict of interest myself – I'm interested in hobby programming for vintage computers – but my understanding comes from Unsloth's fine-tuning instructions. [1]
[1] https://docs.unsloth.ai/basics/datasets-guide https://docs.unsloth.ai/basics/datasets-guide
- rybosome 1y agoNo problem - although I'm out of that particular role, it's appropriate to discuss since the company shared these details already in an openAI press release a few months back. I fine-tuned reasoning models (o1-mini and o3-mini) which were already well into instruction-following and reasoning behavior. The dataset I prepared was taking this into account, but it was just simple prompt/response pairs. Defining the task tightly, ensuring the dataset was of high quality, picking the right hyper parameters, and preparing the proper reward function (and modeling that against the API provided) were the keys to success.
- rbanffy 1y agoThat’s really cool. I’d love to see that process from up close.