3 ms·
> These assistants should be able to make predictions about program _traces_, not just program source text. Are you sure 'traces' is the right word? Not somet
by eru 3y ago
> These assistants should be able to make predictions about program _traces_, not just program source text.
Are you sure 'traces' is the right word? Not something more like ASTs?
Btw, the predict-next-token approach has the benefit of also being able to deal with broken code.
You might also want to compare http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html and https://gwern.net/scaling-hypothesis https://gwern.net/scaling-hypothesis with your idea of adding more domain specific knowledge.