3 ms·

tldr how does it compare to codepilot/gpt4?

• bavell 3y ago
  From the summary: "We perform the most comprehensive evaluation of Code LLMs to date and show that StarCoderBase outperforms every open Code LLM that supports multiple programming languages and matches or outperforms the OpenAI code-cushman-001 model." So I'd assume not up to par with gpt4 or copilot. Can't wait to see it evolve from here!
  • tulip4attoo 3y ago
    GPT4 is ways ahead. On HumanEval, it gets 67%, almost double this one.