4 ms·
I'm not saying its better than 70B, just that its very strong from what others are saying. Actually I am testing the 34B myself (not the 7B), and it seems good
by brucethemoose2 3y ago
I'm not saying its better than 70B, just that its very strong from what others are saying.
Actually I am testing the 34B myself (not the 7B), and it seems good.
- fblgit 3y agoUNA: Uniform Neural Alignment. Haven't u noticed yet? Each model that I uniform, behaves like a pre-trained.. and you likely can fine-tune it again without damaging it. If you chatted with them, you know .. that strange sensation, you know what is it.. Intelligence. Xaberius-34B is the highest performer of the board, and is NOT contaminated.
- valine 3y agoHow much data do you need for UNA? Is a typical fine tuning dataset needed or can you get away with less than that?
- fblgit 3y agodoesn't require much data, in a 7B can take a couple hours ~
- valine 3y agoThat’s cool. A couple hours on a single GPU or like 8x a100s?
- brucethemoose2 3y agoIn addition to what was said, if its anything like DPO you don't need a lot of data, just a good set. For instance, DPO requires "good" and "bad" responses for each given prompt.
- nabakin 3y ago> I'm not saying its better than 70B, just that its very strong from what others are saying. Gotcha > Actually I am testing the 34B myself (not the 7B), and it seems good. I've heard good things about it