3 ms·
This got me curious, so I looked into it. The only post ChatGPT dataset was MOSS-3-SFT, which was used for dialogue fine tuning. MOSS-3 was generated using Chat
by causalmodels 3y ago
This got me curious, so I looked into it. The only post ChatGPT dataset was MOSS-3-SFT, which was used for dialogue fine tuning. MOSS-3 was generated using ChatGPT-3.5-turbo [1]. They explicitly call out their fine tuning data in footnote 13 so I wouldn't say it is faked per say, but it isn't a great look.
[1] https://huggingface.co/fnlp/moss-moon-003-base#data https://huggingface.co/fnlp/moss-moon-003-base#data
- nicklecompte 3y agoGot it, thanks for looking into it more closely. I didn't actually finish reading the paper, and missed that this was in section 3: "Subsequent fine-tuning was conducted on the MOSS finetune dataset [36], which is tailored for dialogue systems." Agreed this almost certainly isn't faked, but the validity of using this dataset at all seems questionable.