3 ms·
They do actually have custom models for autocomplete (which requires very low latency) and applying edits from the LLM (which turns out to require another LLM s
by tomduncalf 2y ago
They do actually have custom models for autocomplete (which requires very low latency) and applying edits from the LLM (which turns out to require another LLM step, as they can’t reliably output perfect diffs)