Compaction, busting the cache, and other issues like that will lose significance when you're running at 15k tokens per second like chatjimmy. Very interesting t

Compaction, busting the cache, and other issues like that will lose significance when you're running at 15k tokens per second like chatjimmy. Very interesting to think about what will change in the future.