2 ms·
You should check out our TerminalBench2.1 score for that, the tasks there can run up to 4h! We got almost no loss (we got Base 66.74% vs Swift 65.84%) with -38.
by kisjovan 19d ago
You should check out our TerminalBench2.1 score for that, the tasks there can run up to 4h! We got almost no loss (we got Base 66.74% vs Swift 65.84%) with -38.7% thinking token reduction. There's also quite a few independent evals on the reddit link as well.
Please let me know when you try the model and if I can help you set it up :)