2 ms·
but have you compared whether or not this strategy is net beneficial? for instance, similar reliable output first time?
by andsoitis 4mo ago
but have you compared whether or not this strategy is net beneficial? for instance, similar reliable output first time?
- verdverm 4mo agoIt is has been shown many times you can use a large model for tip level orchestration or planning, and then have smaller models do exploration and file editing (e.g. in coding). This keeps a bunch of (wasted) tokens out of the big vllm context (e.g. reading a file only to find out it has nothing relevant for the task)