3 ms·
Eventually this specific issue will be RLHF’d out of existence. For now that should mostly solve the problem, but these models aren’t perfect at following instr
by teaearlgraycold 9mo ago
Eventually this specific issue will be RLHF’d out of existence. For now that should mostly solve the problem, but these models aren’t perfect at following instructions. Especially when you’re deep into the context window.
- girvo 9mo ago> Especially when you’re deep into the context window. Though that is, at least to me, a bit of an anti-pattern for exactly that reason. I've found it far more successful to blow away the context and restart with a new prompt from the old context instead of having a very long running back-and-forward. Its better than it was with the latest models, I can have them stick around longer, but it's still a useful pattern to use even with 4.6/5.3
- teaearlgraycold 9mo agoOpus has also clearly been trained to clear the context fairly often through the plan/code/plan cycle.
- catlifeonmars 9mo agoIs it training, or prompting from the CLI?