4 ms·
Thank you for the reply. Maybe I'm daft or out of the loop, I don't see the difference between this method and what I have seen previously as "fine tuning on sy
by nick3443 2y ago
Thank you for the reply. Maybe I'm daft or out of the loop, I don't see the difference between this method and what I have seen previously as "fine tuning on synthetic data".
They also mention in the source tweet that the quality of the base model is important, which is not really accounted for in the $30 figure.
- whimsicalism 2y ago'finetuning on synthetic data' = same LM task with BCE objective, ie. training on next word prediction on good text. 'rl on sparse rewards' = RL objective - reward is given by deterministic evaluation, reward is passed back as sparse signal. there is no 'ground truth' data to compare against.
- nick3443 2y agoAh now I see the difference. Thank you!!