10 ms·

"causing" -> "costing", right?

• vorticalbox 1mo ago
  In this case yeah. If it’s not reading the cache then it has to compute all the context window again and not just the newest tokens.