3 ms·
Does anyone have a write up of the response format the llm uses in these type of editors? I'm assuming the llm can't generate exact diff format without making
by kristianp 2y ago
Does anyone have a write up of the response format the llm uses in these type of editors? I'm assuming the llm can't generate exact diff format without making errors.
- simonw 2y agoYeah, diffs are unreliable. A couple of interesting things to read: 1. This reverse-engineering of VS Code GitHub Copilot - it's a bit old but still very relevant to understanding what's going on under the hood: https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals.html https://thakkarparth007.github.io/copilot-explorer/posts/cop... 2. This post from Aider (a very sophisticated CLI-based coding assistant tool) talking about how the benchmarked different editing formats: https://aider.chat/2023/07/02/benchmarks.html#edit-formats https://aider.chat/2023/07/02/benchmarks.html#edit-formats - and this later post as well, about how diffs without line numbers can work well: https://aider.chat/2023/12/21/unified-diffs.html#unified-diff-editing-format https://aider.chat/2023/12/21/unified-diffs.html#unified-dif... This whole field is still very much under active development, so I'm sure there are lots of other interesting techniques out there.
- kristianp 2y agoThank you, I went looking for how the original copilot worked a few months ago, but didn't find it.