3 ms·

Now they can do partial credit assignment. You use an LLM to evaluate the whole trajectory, pin point what the model did right, what it did wrong, where it too

Now they can do partial credit assignment. You use an LLM to evaluate the whole trajectory, pin point what the model did right, what it did wrong, where it took the wrong path, even re-run from that point. You can get much more than a single numeric score these days from a run.

amluto 3mo ago
The “re-run” part might be rather awkward when your model is poking outside its sandbox to pwn someone else’s system :)