Actually, selectively deleting facts like that does kinda work, essentially what you're doing is scolding-based reinforcement learning. For decent speed you'd w

Actually, selectively deleting facts like that does kinda work, essentially what you're doing is scolding-based reinforcement learning. For decent speed you'd want to probably use variance-reduction techniques like https://arxiv.org/abs/2205.08253 https://arxiv.org/abs/2205.08253 , though. Or a related (would be linked as cited by/cited in the linked when you look on semantic scholar) method that doesn't compute a hessian-vector product (which apparently interacts poorly with the non-linearity of the multi-layer perceptrons as provided by LLM-typical ReLU), in exchange for e.g. sizable batching (gradient accumulation!) at occasional checkpoints.