3 ms·

I think we need a different strategy to instruction tuning for coding LLMs. I don't think StarCoderBase is instruction-tuned off the bat but would serve as a g

I think we need a different strategy to instruction tuning for coding LLMs. I don't think StarCoderBase is instruction-tuned off the bat but would serve as a good starting point for a new technique. RLHF is fine for things that are hard to measure and evaluate, but code is runnable and testable. I propose we try Reinforcement Learning Machine Feedback or RLMF. Prompts and responses are evaluated by how accurate the response evaluated to. We can then train a reward model to help refine StarCoder.

grandmczeb 3y ago
Good idea, but pretty sure this is already widely done. For example, Alex Gravely (architect behind copilot) mentioned on No Priors[1] they would generate the implementations for tests in random github projects and check if they passed as feedback. [1] https://open.spotify.com/episode/2a8Rtm4mhjzennOoAByFKx https://open.spotify.com/episode/2a8Rtm4mhjzennOoAByFKx around 15:10