3 ms·
I just looked on Huggingface.co, and the training data is there. For example, 3.3 Tbyte for code reasoning, 4.5 Tbyte for mathematical reasoning, 8.4 Tbyte of
by adrian_b 1mo ago
I just looked on Huggingface.co, and the training data is there.
For example, 3.3 Tbyte for code reasoning, 4.5 Tbyte for mathematical reasoning, 8.4 Tbyte of pre-train behaviors, and so on.
I did not compute the sum of the dataset sizes, but it appears to be some tens of Tbyte. Nonetheless, I assume that this amount of training data is more than an order of magnitude less than what OpenAI, Anthropic and the like have used, which must have been at least many hundreds of Tbyte, but more likely several thousands of Tbyte of data.
- luciana1u 1mo ago[flagged]
- lambda 1mo agoLooks like we're still waiting on that, they have placeholder repos but haven't populated them yet: * https://github.com/ifm-ai/xllm https://github.com/ifm-ai/xllm * https://github.com/ifm-ai/horizon-post-train https://github.com/ifm-ai/horizon-post-train Their previous model, K2 Think V2, was release with fully open training data and recipe, so I would imagine that they are committed to that, but yeah, the repos for this new model are still just placeholders. * https://mbzuai.ac.ae/news/k2-think-v2-a-fully-sovereign-reasoning-model/ https://mbzuai.ac.ae/news/k2-think-v2-a-fully-sovereign-reas... * https://github.com/LLM360/Reasoning360 https://github.com/LLM360/Reasoning360
- luciana1u 1mo ago[flagged]
- lambda 1mo agoWeights are up: https://huggingface.co/collections/IFM/k2-horizon https://huggingface.co/collections/IFM/k2-horizon It's the training code that is not up yet, but this group has a history of publishing code so I would expect it, though of course you can never count on it until posted.