3 ms·

you lose coherence across chunks of context size. i wish i could spend compute to pre-train on some knowledge.