13 ms·

Reposting my commentary: The TL;DR is that this appears to beat both OpenAI Codex (12 billion params) and DeepMind's AlphaCode (1 billion params), despite havi

by moyix 4y ago

Reposting my commentary: The TL;DR is that this appears to beat both OpenAI Codex (12 billion params) and DeepMind's AlphaCode (1 billion params), despite having only 770M parameters. There is also a paper: https://arxiv.org/abs/2207.01780 https://arxiv.org/abs/2207.01780 And the code is available: https://github.com/salesforce/CodeRL https://github.com/salesforce/CodeRL So this seems to buck the trend of needing enormous models, by being smarter about taking into account the output from running the AI-generated program on test cases.

algo_trader 4y ago
RL is an "obvious" extension to code generating LLMs Try to compile, try to run, estimate reward, iterate. Possibly there is a cost issue, because sampling LLMs is more expensive than running starcraft?!
algo_trader 4y ago
EDIT: ha! turns out I am following you on twitter https://twitter.com/moyix/status/1551932519796408321 https://twitter.com/moyix/status/1551932519796408321 Please release this as a docker or something.. Ask the AI to do it for you
- moyix 4y ago
  I'm working on integrating it with a VSCode plugin right now :) After that I'll write up a post and try to release it as something easy to use!
a-dub 4y ago
> So this seems to buck the trend of needing enormous models, by being smarter about taking into account the output from running the AI-generated program on test cases. i'd be curious if it finds degenerate solutions and what they look like when the test cases aren't conceptually complete...
PeterisP 4y ago
> the trend of needing enormous models I would rather say that there isn't a trend of needing enormous models but rather a trend of observing that enormous models (and straightforward increases in size) work. We've already seen many other tricks applied for transformer models (essentially, most of "BERTology") that demonstrated that being smarter about taking into account some stuff in training can get a big benefit, but that is all orthogonal to the size issue. That trend would be bucked only if it turned out that for this task 770M is sufficient, i.e. that scaling this 'being smarter' approach from 770M parameters to the 12B parameters of OpenAI Codex does not get a further improvement - however, I am quite convinced that it would be noticeably better than this; and my expectations would be that the eventual outcome of this technique would be that OpenAI Codex v2 (or however that would be called) would beat this by having a model that includes this technique and some other improvements to training process and increases the size beyond 12B parameters.

by moyix 4y ago

algo_trader 4y ago

RL is an "obvious" extension to code generating LLMs Try to compile, try to run, estimate reward, iterate. Possibly there is a cost issue, because sampling LLMs is more expensive than running starcraft?!

algo_trader 4y ago

EDIT: ha! turns out I am following you on twitter https://twitter.com/moyix/status/1551932519796408321 https://twitter.com/moyix/status/1551932519796408321 Please release this as a docker or something.. Ask the AI to do it for you

moyix 4y ago
I'm working on integrating it with a VSCode plugin right now :) After that I'll write up a post and try to release it as something easy to use!

a-dub 4y ago

> So this seems to buck the trend of needing enormous models, by being smarter about taking into account the output from running the AI-generated program on test cases. i'd be curious if it finds degenerate solutions and what they look like when the test cases aren't conceptually complete...

PeterisP 4y ago

> the trend of needing enormous models I would rather say that there isn't a trend of needing enormous models but rather a trend of observing that enormous models (and straightforward increases in size) work. We've already seen many other tricks applied for transformer models (essentially, most of "BERTology") that demonstrated that being smarter about taking into account some stuff in training can get a big benefit, but that is all orthogonal to the size issue. That trend would be bucked only if it turned out that for this task 770M is sufficient, i.e. that scaling this 'being smarter' approach from 770M parameters to the 12B parameters of OpenAI Codex does not get a further improvement - however, I am quite convinced that it would be noticeably better than this; and my expectations would be that the eventual outcome of this technique would be that OpenAI Codex v2 (or however that would be called) would beat this by having a model that includes this technique and some other improvements to training process and increases the size beyond 12B parameters.