4 ms·
Oh NVIDIA released an open weights 340 billion parameter LLM! It should be the biggest open weights to date I think (Grok 314b). It's trained on 8 trillion to
by danielhanchen 2y ago
Oh NVIDIA released an open weights 340 billion parameter LLM!
It should be the biggest open weights to date I think (Grok 314b).
It's trained on 8 trillion tokens, and some benchmarks show it does better than or equal to GPT-4o!
They released 3 checkpoints - the base, the instruct and a reward aligned model.
See https://huggingface.co/collections/nvidia/nemotron-4-340b-666b7ebaf1b3867caf2f1911 https://huggingface.co/collections/nvidia/nemotron-4-340b-66... for all the checkpoints