There were actually multiple bugs which impacted long context benchmarks and general inference - I helped fix some of them. 1. RMS norm eps was 1e-6, but shoul

There were actually multiple bugs which impacted long context benchmarks and general inference - I helped fix some of them. 1. RMS norm eps was 1e-6, but should be 1e-5 - see https://github.com/huggingface/transformers/pull/37418 https://github.com/huggingface/transformers/pull/37418 2. Llama 4 Scout changed RoPE settings after release - conversion script for llama.cpp had to be fixed. See https://github.com/ggml-org/llama.cpp/pull/12889 https://github.com/ggml-org/llama.cpp/pull/12889 3. vLLM and the Llama 4 team found QK Norm was normalizing across entire Q & K which was wrong - accuracy increased by 2%. See https://github.com/vllm-project/vllm/pull/16311 https://github.com/vllm-project/vllm/pull/16311 If you see https://x.com/WolframRvnwlf/status/1909735579564331016 https://x.com/WolframRvnwlf/status/1909735579564331016 - the GGUFs I uploaded for Scout actually did better than inference providers by +~5% on MMLU Pro. https://docs.unsloth.ai/basics/tutorial-how-to-run-and-fine-tune-llama-4 https://docs.unsloth.ai/basics/tutorial-how-to-run-and-fine-... has more details