3 ms·
How are they going to double check without incurring the cost of running it again?
by throwaway314155 2y ago
How are they going to double check without incurring the cost of running it again?
- geor9e 2y agoStart with a regex (or fast tiny model) to flag obvious time-sensitive tasks. Else, do the task early by prompting it "if this requires up to the minute information, output cancel, else [prompt]". At best, it's 1 regex + 1 full inference. At worst, it's 1 regex + 1 output token + 1 full inference.