4 ms·
WSJ is wrong. Autocomplete code models are small (5B param) and very cheap. A single inference is roughly $0.000001.
by ipsum2 2y ago
WSJ is wrong. Autocomplete code models are small (5B param) and very cheap. A single inference is roughly $0.000001.
- acchow 2y agoCopilot uses GPT4 now
- ipsum2 2y agoI mentioned autocomplete, not the chat service.
- SgtBastard 2y agoGPT4 isn’t a chat service, friend.
- acka 2y agoI think the point that ipsum2 is trying to make is that Copilot's chat service and its code completion service could be using different models, which is not uncommon for coding assistants. Continue[0] for example can use up to 3 different models in a session: a large model such as GPT-4 Turbo for chat and code QA, a smaller low latency model such as StarCoder2-3B for code completion, and yet another model such as all-MiniLM-L6-v2 for generating embeddings. [0] https://www.continue.dev/ https://www.continue.dev/