3 ms·
It’s not supposed to infer beyond max seq len right now, it’s undefined behavior. It’s possible to fix just have to think it through a bit because of RoPE, whic
by karpathy 3y ago
It’s not supposed to infer beyond max seq len right now, it’s undefined behavior. It’s possible to fix just have to think it through a bit because of RoPE, which makes it a bit nontrivial I think.