4 ms·

I think you got that wrong; Scott wrote the terms. (He wrote the comment [1] with those rules.) Someone in the comments agreed to them. Then he changed the ter

I think you got that wrong; Scott wrote the terms. (He wrote the comment [1] with those rules.) Someone in the comments agreed to them. Then he changed the terms because Imagen won't do people. I think that's cheating. [1] https://astralcodexten.substack.com/p/a-guide-to-asking-robots-to-design/comment/6945486 https://astralcodexten.substack.com/p/a-guide-to-asking-robo...

cwillu 4y ago
Again, if the counter-party agrees to the terms and the changes, how is it cheating?
- skybrian 4y ago
  It's not clear whether the counter-party agreed to the change. See: https://news.ycombinator.com/item?id=32858426 https://news.ycombinator.com/item?id=32858426
lalaithion 4y ago
I think you missed the point of my comment. Yes, Scott wrote the comment containing that proposal. But my point was that it was an agreement. Two people who disagreed about AI agreed on the rules, so you can't accuse one of them of being unfair because you don't like the rules. Sure, you can say "that's a bad bet, Scott will obviously win", but you can't say "He shouldn’t declare victory after cherry-picking good results from a small sample of questions", because those terms were explicitly set in advance. The humans -> robots change is possibly dubious, yes. I don't think that it's super important, but if it were me, I wouldn't have posted the blog post as is. I would have waited until some AI passed all the prompts with humans, like it most certainly will in a year.
- telotortium 4y ago
  Stable Diffusion will soon update to use the biggest CLIP model in existence, which may improve understanding of composition: https://news.ycombinator.com/edit?id=32858809 https://news.ycombinator.com/edit?id=32858809
  - TOMDM 4y ago
    Is it the largest CLIP model or the largest open source CLIP model?
    - dhruval 4y ago
      The latter
- mannykannot 4y ago
  I feel your point, in turn, misses the point of the article. Yes, given that someone accepted the terms and those terms were met, then Alexander won the bet, no question. That particular fact about the bet, however, does nothing to counter Marcus' criticisms of Alexander's methodology and his claims of significant progress on the compositionality problem.
  - thaumasiotes 4y ago
    > Yes, given that someone accepted the terms and those terms were met, then Alexander won the bet, no question. You didn't read his post declaring victory. There's plenty of question; he's giving credit for "a llama with a bell on its tail" to ten pictures of llamas without bells on their tails, and for "a robot farmer" to ten pictures of robots with absolutely nothing to suggest they might be farmers. He was way, way too eager to believe that he'd won.
    - mannykannot 4y ago
      You prompted me to look more closely at the terms of the bet [1], and they are indeed absurdly biased: just one in ten on three of the five scenarios counts as success. On the substitution of a robot, which simplifies the task, Alexander says "we" agreed to it, and I assume the "we" includes Vitor, as, in his comment (in which he does not concede defeat), he seems to accept the substitution (he also acknowledges that he probably should not have accepted these terms.) The other issue is who judges the outcome. The terms specify Gwern or Cassander (without having secured the assent of either) or they will "figure something out." In his victory claim, Alexander does not mention any independent judge, and interestingly, Gwern posted two comments without explicitly concurring with Alexander's claim, though his second comment might be read as tacitly accepting it. My initial comment, therefore, needs some modification: replace the "given that" with "if", and I think it stands as a counterfactual conditional, having a probably-false antecedent. I don't think Alexander is doing his reputation any favors by being so triumphalist about this misbegotten bet. [1] https://astralcodexten.substack.com/p/a-guide-to-asking-robots-to-design/comment/6945486 https://astralcodexten.substack.com/p/a-guide-to-asking-robo...
SteveDR 4y ago
Cheating? Thatd make sense if the bet were about the future of products and ethics. Weren’t they trying to predict the future of the state of the art technology?
- skybrian 4y ago
  It depends on what you mean by "technology" and "exists." A research project at Google intentionally won't render people. Maybe it could render people, theoretically, but without evidence, we don't know how well.

cwillu 4y ago

Again, if the counter-party agrees to the terms and the changes, how is it cheating?

skybrian 4y ago
It's not clear whether the counter-party agreed to the change. See: https://news.ycombinator.com/item?id=32858426 https://news.ycombinator.com/item?id=32858426

lalaithion 4y ago

I think you missed the point of my comment. Yes, Scott wrote the comment containing that proposal. But my point was that it was an agreement. Two people who disagreed about AI agreed on the rules, so you can't accuse one of them of being unfair because you don't like the rules. Sure, you can say "that's a bad bet, Scott will obviously win", but you can't say "He shouldn’t declare victory after cherry-picking good results from a small sample of questions", because those terms were explicitly set in advance. The humans -> robots change is possibly dubious, yes. I don't think that it's super important, but if it were me, I wouldn't have posted the blog post as is. I would have waited until some AI passed all the prompts with humans, like it most certainly will in a year.

telotortium 4y ago
Stable Diffusion will soon update to use the biggest CLIP model in existence, which may improve understanding of composition: https://news.ycombinator.com/edit?id=32858809 https://news.ycombinator.com/edit?id=32858809
- TOMDM 4y ago
  Is it the largest CLIP model or the largest open source CLIP model?
  - dhruval 4y ago
    The latter
mannykannot 4y ago
I feel your point, in turn, misses the point of the article. Yes, given that someone accepted the terms and those terms were met, then Alexander won the bet, no question. That particular fact about the bet, however, does nothing to counter Marcus' criticisms of Alexander's methodology and his claims of significant progress on the compositionality problem.
- thaumasiotes 4y ago
  > Yes, given that someone accepted the terms and those terms were met, then Alexander won the bet, no question. You didn't read his post declaring victory. There's plenty of question; he's giving credit for "a llama with a bell on its tail" to ten pictures of llamas without bells on their tails, and for "a robot farmer" to ten pictures of robots with absolutely nothing to suggest they might be farmers. He was way, way too eager to believe that he'd won.
  - mannykannot 4y ago
    You prompted me to look more closely at the terms of the bet [1], and they are indeed absurdly biased: just one in ten on three of the five scenarios counts as success. On the substitution of a robot, which simplifies the task, Alexander says "we" agreed to it, and I assume the "we" includes Vitor, as, in his comment (in which he does not concede defeat), he seems to accept the substitution (he also acknowledges that he probably should not have accepted these terms.) The other issue is who judges the outcome. The terms specify Gwern or Cassander (without having secured the assent of either) or they will "figure something out." In his victory claim, Alexander does not mention any independent judge, and interestingly, Gwern posted two comments without explicitly concurring with Alexander's claim, though his second comment might be read as tacitly accepting it. My initial comment, therefore, needs some modification: replace the "given that" with "if", and I think it stands as a counterfactual conditional, having a probably-false antecedent. I don't think Alexander is doing his reputation any favors by being so triumphalist about this misbegotten bet. [1] https://astralcodexten.substack.com/p/a-guide-to-asking-robots-to-design/comment/6945486 https://astralcodexten.substack.com/p/a-guide-to-asking-robo...

SteveDR 4y ago

Cheating? Thatd make sense if the bet were about the future of products and ethics. Weren’t they trying to predict the future of the state of the art technology?

skybrian 4y ago
It depends on what you mean by "technology" and "exists." A research project at Google intentionally won't render people. Maybe it could render people, theoretically, but without evidence, we don't know how well.