3 ms·
very simlar experience. I would also add that I run it this way ~12hour a day non-stop. 300M / tokens per day (99.7% cache hit).
by 5701652400 3mo ago
very simlar experience.
I would also add that I run it this way ~12hour a day non-stop. 300M / tokens per day (99.7% cache hit).