3 ms·
I propose four training stages: 1. Train without pauses for fast response 2. Train with variable numbers of inserted pauses, so the model can take more time (
by Nevermark 3y ago
I propose four training stages:
1. Train without pauses for fast response
2. Train with variable numbers of inserted pauses, so the model can take more time (computation) to improve its performance.
Where pauses are represented both by pause tokens, and a new pause countdown element.
I.e:
Sequence:
…, token, pause, pause, pause, token, …
Pause element:
…, 0, 3, 2, 1, 0, …
3. Train the model model to generate its own next pause value (insert its own pauses), to optimize a meta-performance tradeoff function that ranges from the regular performance measure (independent of delay) vs. delay.
The tradeoff is based on an “urgency” value between 0 and 1, that adjusts the tradeoff meta-performance weighting between just delay-independent accuracy vs. just speed.
The urgency is used at each step to calculate the tradeoff meta-performance, and as a new urgency input element so the model knows what tradeoff level it is supposed to optimize.
4. Train the model to generate its own urgency values, based on examples of prompts which either communicate urgency based on request or context.
“Quick, give me an estimated value for …”
“The reactor is about to explode, what are the shutdown codes?”
Note, that the model generated urgency value and the next urgency input don’t have to be the same.
The generated urgency could be pushed higher or lower algorithmically, to get the next urgency input, if the model is being used in a context that has its own urgency relevant information.
—-
So many good ideas. It’s clear models are going to improve quickly.
Next step beyond a model that can leverage pauses for better performance, and a model that controls its own pauses, is a model that can be interrupted.
Interruptions could be urgency related, cancellations, modifications, or irrelevant:
“Hurry up”
“Forget that, do this other thing.”
“Oh, and the answer should be odd.”
“While you think, I am enjoying a nice cup of tea.”