3 ms·

I didn't get idea of this benchmark, they ask to produce code to replicate result of papers, which already have code on github?..

eightysixfour 2y ago
You don’t see the value of independent replication of findings? The agent didn’t have access to the code, although they acknowledge it could theoretically be in the training set, even then the original code wouldn’t conform to the structure of the test.
- riku_iki 2y ago
  > even then the original code wouldn’t conform to the structure of the test. yeah, this part should be central in this work: how well those tests are built, do they actually are catching data leakage, how this is measured, etc.
kelseyfrog 2y ago
The point is to bootstrap self-improving AI. Once a measurement becomes a goal, model makers target saturating it. There is a coefficient of intelligence replication ie: Model M with intelligence I_m, can reproduce a model N with intelligence I_n. When (I_n / I_m) > 1 we'll have a runaway intelligence explosion. There are of course several elements in the chain - akin to the Drake equation for intelligent machines - and their combined multiplicative effect determines the overall intelligence of the system. If f(paper) -> code is the weakest part of the chain, it makes sense to target that.
- riku_iki 2y ago
  > If f(paper) -> code is the weakest part of the chain, it makes sense to target that. my point is that LLMs are already potentially seeing solution on github, so you can't use that benchmark as metric unless there is some explanation.
  - kelseyfrog 2y ago
    How does that work with knowledge cutoff?
    - riku_iki 2y ago
      It could work with knowledge cut off if they can reliably guarantee it, and also make sure LLMs are not searching github under the surface.
      - kelseyfrog 2y ago
        What's the likelihood that the researchers have done this? It seems fairly easy.
        riku_iki 2y ago
        I honestly have no idea how OAI researchers can guarantee cut off date for Antropic models for example.

eightysixfour 2y ago

You don’t see the value of independent replication of findings? The agent didn’t have access to the code, although they acknowledge it could theoretically be in the training set, even then the original code wouldn’t conform to the structure of the test.

riku_iki 2y ago
> even then the original code wouldn’t conform to the structure of the test. yeah, this part should be central in this work: how well those tests are built, do they actually are catching data leakage, how this is measured, etc.

kelseyfrog 2y ago

The point is to bootstrap self-improving AI. Once a measurement becomes a goal, model makers target saturating it. There is a coefficient of intelligence replication ie: Model M with intelligence I_m, can reproduce a model N with intelligence I_n. When (I_n / I_m) > 1 we'll have a runaway intelligence explosion. There are of course several elements in the chain - akin to the Drake equation for intelligent machines - and their combined multiplicative effect determines the overall intelligence of the system. If f(paper) -> code is the weakest part of the chain, it makes sense to target that.

riku_iki 2y ago
> If f(paper) -> code is the weakest part of the chain, it makes sense to target that. my point is that LLMs are already potentially seeing solution on github, so you can't use that benchmark as metric unless there is some explanation.
- kelseyfrog 2y ago
  How does that work with knowledge cutoff?
  - riku_iki 2y ago
    It could work with knowledge cut off if they can reliably guarantee it, and also make sure LLMs are not searching github under the surface.
    - kelseyfrog 2y ago
      What's the likelihood that the researchers have done this? It seems fairly easy.
      - riku_iki 2y ago
        I honestly have no idea how OAI researchers can guarantee cut off date for Antropic models for example.