thinker567

3 ms·

1. 1.▲
   New synthetic dataset beating MSFT and mistral's SFT recipe(huggingface.co)

   3 points by thinker567 2y ago|0 comments
2. 2.▲
   by thinker567 2y ago

   They didn’t say they beat the 3B, their 1.7B outperforms llama3.2 1B thanks to 11T tokens of high quality data (Hugging Face are the ones behind FineWeb dataset that everyone uses now). Btw Qwen2.5-1B also surpasses llama3.2 1B by a large m