2 ms·
> our velocity has only gone up That is super curious - using more low quality cheaper models increased your velocity? My prior would have been slightly reduc
by jmalicki 4mo ago
> our velocity has only gone up
That is super curious - using more low quality cheaper models increased your velocity? My prior would have been slightly reduced velocity but massive reduction in token costs made it worthwhile.
Is that due to the faster inference time?