6 ms·
Really cool work, you get the actual traces. Looks like the vendors can all reliably fix this one though. A harder to defend against approach here where they w
by nervai 2mo ago
Really cool work, you get the actual traces. Looks like the vendors can all reliably fix this one though.
A harder to defend against approach here where they work backwards from the results and ask the model to generate a plausible trace:
How to Steal Reasoning Without Reasoning Traces
https://arxiv.org/pdf/2603.07267 https://arxiv.org/pdf/2603.07267
- dannyw 2mo agoTrace Inversion is fascinating, but it’s more of an independent reconstruction that will give you some coherent-looking generated CoT; but not necessarily anywhere close or related to the underlying model’s CoT.
- nervai 2mo agoI didn't read the paper in details but they claim there is high overlap between the synthetic traces and the ground truth ones (not sure how they confirmed that for blackbox models though, I guess they must have compared to open source models). They also talk about successful distillation of black box model capabilities with the approach.