4 ms·Context length performance and memory scales N^2. Smaller tokens mean worse scaling, up to a point.by cma 2y agoContext length performance and memory scales N^2. Smaller tokens mean worse scaling, up to a point.