3 ms·
The recent DeepMind paper on FunSearch highlighted their use of pre-trained large language models (LLMs) for generating code improvements. Interestingly, while
by JoelJacobson 3y ago
The recent DeepMind paper on FunSearch highlighted their use of pre-trained large language models (LLMs) for generating code improvements. Interestingly, while the main LLM used was Codey, built on the PaLM2 model family, they also referenced StarCoder, an open-source LLM, in their supplementary information.
However, the GitHub repository for FunSearch doesn't include implementations for these LLMs. For instance, in `sampler.py`:
```
class LLM:
"""Language model that predicts continuation of provided source code."""
def __init__(self, samples_per_prompt: int) -> None:
self._samples_per_prompt = samples_per_prompt
def _draw_sample(self, prompt: str) -> str:
"""Returns a predicted continuation of `prompt`."""
raise NotImplementedError('Must provide a language model.')
```
This code suggests the need for an external LLM implementation. Given that they successfully used StarCoder, it's surprising that no integration guide or basic implementation for it (or any similar open-source LLM) is provided. Such an inclusion would have significantly enhanced the reproducibility and accessibility of their research.