4 ms·Forgot one: the positional encoding also changed, llama3 uses RoPE, gpt2 uses a learned embedding.by jacobn 2y agoForgot one: the positional encoding also changed, llama3 uses RoPE, gpt2 uses a learned embedding.