We hit the same problem building a voice-memo-to-notes tool: "meet Tuesday, no wait, Wednesday" is exactly the failure mode. We ended up not trying to fix it at

We hit the same problem building a voice-memo-to-notes tool: "meet Tuesday, no wait, Wednesday" is exactly the failure mode. We ended up not trying to fix it at the ASR layer at all. We run faster-whisper (large-v3, local) and let the raw transcript keep the self-correction intact, including the discarded "Tuesday." The summarization pass resolves it, because it has document-level context the transcriber doesn't, and it's cheap to re-run on a short transcript. A technically "wrong" transcript still produces the right structured output most of the time. The failure mode shifts rather than disappears though: on longer memos where corrections stack up, the summarizer occasionally keeps the wrong one, and that's much harder to catch than a raw WER miss because the sentence still reads fine. Do you know of anyone scoring disfluency-recovery as its own metric instead of folding it into WER? Feels like exactly the kind of thing that would need a downstream-task-aware eval rather than a transcript-only one.