3 ms·

The original question was “why would they use an LLM here”. Whether candidates are an entire program or just a portion of the program is immaterial to that ques

The original question was “why would they use an LLM here”. Whether candidates are an entire program or just a portion of the program is immaterial to that question. In fact, it is an example of a situation where random decoding is especially poorly matched against an autoregressive model, since there’s already context to use. Anyways, you can see there is a cold start issue in the ablation study since the random strategy didn’t even begin to register until about halfway through the test.

YeGoblynQueenne 3y ago
Of course if you start without any inductive biases (i.e. a program schema in the case of the FunSearch approach) there will be a "cold start" problem, but FunSearch doesn't start cold, as genetic programming doesn't start cold. The question is the "cold start issue". That's what I'm addressing. Maybe you're not used to being corrected? Get used to it.