3 ms·
The instruct tuning can be done with several open datasets at minimal cost. Should be easy for someone to create their own open model.
by dougmwne 4y ago
The instruct tuning can be done with several open datasets at minimal cost. Should be easy for someone to create their own open model.
- jacooper 4y agoHow?
- dougmwne 4y agohttps://github.com/tatsu-lab/stanford_alpaca https://github.com/tatsu-lab/stanford_alpaca
- MallocVoidstar 4y agoThat dataset is licensed under CC BY NC 4.0, which is not open. It also has a bunch of garbage in it; see https://github.com/gururise/AlpacaDataCleaned https://github.com/gururise/AlpacaDataCleaned
- dougmwne 4y agoI wonder what happens if you just feel that dataset back into another LLM to re-write it and filter out the low quality items? IS there still any connection to the original copyright? How would that even be proven?
- MacsHeadroom 4y agoYou can finetune 7B in a couple of hours on a $200 3060 with https://github.com/johnsmith0031/alpaca_lora_4bit https://github.com/johnsmith0031/alpaca_lora_4bit