3 ms·
A lot of answers don't go stale for hours or days. They'll do the task early, at an off-peak time, hidden from the user, double-check that it really wasn't time
by geor9e 2y ago
A lot of answers don't go stale for hours or days. They'll do the task early, at an off-peak time, hidden from the user, double-check that it really wasn't time sensitive, then surface the saved answer at the time desired.
- throwaway314155 2y agoHow are they going to double check without incurring the cost of running it again?
- geor9e 2y agoStart with a regex (or fast tiny model) to flag obvious time-sensitive tasks. Else, do the task early by prompting it "if this requires up to the minute information, output cancel, else [prompt]". At best, it's 1 regex + 1 full inference. At worst, it's 1 regex + 1 output token + 1 full inference.