4 ms·
Wouldn't NDCG/token results vary wildly depending on the agent's query and the number of returned items? e.g. agents often run `grep -m 5 "QUERY"` with differe
by esafranchik 5mo ago
Wouldn't NDCG/token results vary wildly depending on the agent's query and the number of returned items?
e.g. agents often run `grep -m 5 "QUERY"` with different queries, instead of one big grep for all items.
- stephantul 5mo agoThe same holds for semble: the agent can fire off many different semble queries with different k/parameters. I guess the point we’re trying to make is that you need fewer semble queries to achieve the same outcome, compared to grep+readfile calls.