5 ms·

Correct title: an unknown author from an unknown startup claims in an arXiv preprint (listed somehow in the cs.AI archive) that Othello has been solved. The pap

Correct title: an unknown author from an unknown startup claims in an arXiv preprint (listed somehow in the cs.AI archive) that Othello has been solved. The paper is 13 pages long. I wish that the superconductor debacle (and many others) had taught people critical thinking and skepticism. And yet this is the #1 post on HN right now.

fbdab103 3y ago
Trust but verify. People were excited, but pretty much the first thing on everyone's mind was looking for replication. In contrast, this is not spouting a revolution in physics, but a mathematical puzzle that is mostly trivia. I suspect that the best computer Othello algorithm can already trounce a human.
- uxp8u61q 3y ago
  > a mathematical puzzle that is mostly trivia "Solving" a game is not "trivia" or a "puzzle". It's an actual mathematical problem that requires nontrivial proofs for nontrivial games. Think about it: a game as "simple" as checkers was only solved in 2007. > I suspect that the best computer Othello algorithm can already trounce a human. That's a completely different problem. Chess computers are much, much better than humans, but chess is far from being solved.
  - sebzim4500 3y ago
    It's not obvious to me that othello is much more complicated than checkers. Admittedly, I haven't played since I was a kid and I don't completely remember what the rules are.
    - WoodenChair 3y ago
      So you don't really know much about the computational complexity of the game and you're saying it's trivial. How hard a game is to "solve" at least using traditional algorithmic techniques is related to its search space. That is a combination of the length of the game and the number of possible moves in each position (ply and branching factor). Chess has more possible positions than exist atoms in the universe by some estimates. Checkers has a tractable, for modern supercomputers, 500 billion possible positions with about a ~6 branching factor over an average of ~50 ply games. Othello is several orders of magnitude more complex than checkers but not quite at the level of chess. So, this is a big claim. It's by no means trivial.
      - kbenson 3y ago
        The person you're responding to wasn't the one that said it was "trivia" (much less trivial). I don't think anyone was saying it's easy, just that they didn't think it was very useful to know.
        WoodenChair 3y ago
        The post I'm replying to said: > It's not obvious to me that othello is much more complicated than checkers. How is that not saying it's trivial when the context is a thread mentioning that Checkers has already been solved.
        kbenson 3y ago
        In the past, "solving" games has often been done through novel changes in hardware and/or approaches and algorithms as I understand it. The person you replied to stated that they didn't know why Othello was an achievement beyond checkers. That doesn't mean it's trivial, but it also doesn't mean it required any additional insight or advances to do so if it's not specifically more complicated than checkers. They then went on to qualify their statement with the information that they don't specifically remember the rules of Othello. In a discussion held in good faith, this should be assumed to be an invitation to add additional insight as to why Othello might be more complicated. You either know the reason why and didn't state it, or believe it's more complicated based on authorities you trust and didn't relate that and instead stated it authoritatively, or don't know it at all and misrepresented it. None of those are very useful for discussion. The former two are easy to rectify by explaining, qualifying, or referring someone to additional info though, and would have been a more useful way to respond. Ultimately though, it sort of feels like you're reaching for an additional interpretation that makes an earlier misinterpretation still valid, when you could just say "oops. Most of my comment still stands as accurate, but I guess I misinterpreted your point." Edit: from looking at you profile, it appears you are a CS professor. Perhaps part of the issue is context. This is a public forum, not a classroom, and while in a classroom your statements can be taken with an assumed authority, nobody necessarily knows that about you here when you post, so some additional qualification of your statements or background may be warranted.
- notfed 3y ago
  > Trust but verify. Ok, so, do you trust everything posted on the Internet until you verify it? I really think "distrust but verify" is a heck of a better rule, unless your source has a really solid track record.
- coldtea 3y ago
  >Trust but verify Given that most papers don't replicate, and most hype ends up BS, "don't trust until it's verified" is a much better rule.
throw101010 3y ago
> And yet this is the #1 post on HN right now. Doesn't it make sense to give some visibility to such claims so they can be verified? I would think a lot of people on HN are equipped to take part in this informally. But I agree with you that the title is too categorical and should be updated.
scythe 3y ago
You really wouldn't want to have a post sit on top of Hacker News for an hour and nothing comes of it. Think of how much we could have gotten done on this messageboard otherwise!
- mcphage 3y ago
  Boy are you gonna feel foolish when the #2 post on HN is “Cancer almost cured”, and you realize how dangerous a diversion this was…
anigbrowl 3y ago
Boo. The paper is cautiously written, addresses a very tractable and well-defined problem, confirms what experts expected, and is straightforward to implement (the source code is on Github). And yet this posturing is the #1 comment on this thread right now.
- notfed 3y ago
  Great, then let's name it "Othello claimed to be solved", then pound it with peer reviews and reproductions for a bit, then we can phrase it as "Othello is solved".
  - SomaticPirate 3y ago
    The source code is on Github? Why not run it yourself and perform the peer review? The implementation is quite straightforward
    - uxp8u61q 3y ago
      You need to prove that the code is correct, that's the tricky part. Otherwise I could just write `return true;` as the whole code and tell people to run it, it's easy, just do the peer review yourself.
cwillu 3y ago
There was absolutely nothing wrong with the superconductor “debacle”. What you're preaching is not skepticism, but cynicism.
- coldtea 3y ago
  Given the myriad posts and articles about "huge breakthrough", "industry changing" and so on, scepticism in that case would translate to way more cynicism.
g9yuayon 3y ago
I thought the post is #1 on HN because people here are genuinely interested in the topic and would like to at least examine the paper and discuss the merits of it.
klyrs 3y ago
I was watching every development of that superconductivity claim. I was quite enthused! But never once did I move beyond "groundbreaking if true." This doesn't even have the potential to be groundbreaking. Get over yourself, let people get excited about little stuff.
- notfed 3y ago
  > Get over yourself, let people get excited about little stuff. Some people are looking for facts and science, and some are looking for enthusiasm and excitement. Can't discount either one, I suppose.
  - CyberDildonics 3y ago
    Can't discount either one, I suppose. People getting each other excited by pretending fiction is reality is called religion. That's fine if people are honest about it.
  - klyrs 3y ago
    You can be enthusiastic and excited while looking for facts and science. In fact, my interest in rooting out truths tends to go hand in hand with my passion for the topic. Negative results are important for scientific advancement, and I was just as excited to see LK99 fully analyzed, despite it not turning out to be the holy grail. We always knew that was a longshot; I enjoyed the journey regardless of the destination. My admonition "get over yourself" is a response to this ridiculous idea that science should be a joyless affair. It isn't, and must not be. Historically, fuddy-duddy downers don't make the big discovaries.
- jmye 3y ago
  > Get over yourself, let people get excited about little stuff. No one has said you’re not allowed to be excited if you want. Just like GP is allowed to be a wet blanket if they want. Speaking of “get over yourself.” I’m so tired of this “if you don’t completely validate and elevate my feelings then you’re bad/mean and preventing me from feeling what I want” crap. God forbid anyone ever disagree with you.
bigbacaloa 3y ago
[dead]
gfodor 3y ago
Calling it a debacle is an admission you yourself could benefit from your own advice.
satoqz 3y ago
I would not call Preferred Networks an "unknown startup" - They've been prominent within the AI research community for quite some time and have contributed important projects such as CuPy and Chainer, the latter introducing the "define-by-run" concept which was later implemented in frameworks such as PyTorch and Tensorflow.

fbdab103 3y ago

Trust but verify. People were excited, but pretty much the first thing on everyone's mind was looking for replication. In contrast, this is not spouting a revolution in physics, but a mathematical puzzle that is mostly trivia. I suspect that the best computer Othello algorithm can already trounce a human.

uxp8u61q 3y ago
> a mathematical puzzle that is mostly trivia "Solving" a game is not "trivia" or a "puzzle". It's an actual mathematical problem that requires nontrivial proofs for nontrivial games. Think about it: a game as "simple" as checkers was only solved in 2007. > I suspect that the best computer Othello algorithm can already trounce a human. That's a completely different problem. Chess computers are much, much better than humans, but chess is far from being solved.
- sebzim4500 3y ago
  It's not obvious to me that othello is much more complicated than checkers. Admittedly, I haven't played since I was a kid and I don't completely remember what the rules are.
  - WoodenChair 3y ago
    So you don't really know much about the computational complexity of the game and you're saying it's trivial. How hard a game is to "solve" at least using traditional algorithmic techniques is related to its search space. That is a combination of the length of the game and the number of possible moves in each position (ply and branching factor). Chess has more possible positions than exist atoms in the universe by some estimates. Checkers has a tractable, for modern supercomputers, 500 billion possible positions with about a ~6 branching factor over an average of ~50 ply games. Othello is several orders of magnitude more complex than checkers but not quite at the level of chess. So, this is a big claim. It's by no means trivial.
    - kbenson 3y ago
      The person you're responding to wasn't the one that said it was "trivia" (much less trivial). I don't think anyone was saying it's easy, just that they didn't think it was very useful to know.
      - WoodenChair 3y ago
        The post I'm replying to said: > It's not obvious to me that othello is much more complicated than checkers. How is that not saying it's trivial when the context is a thread mentioning that Checkers has already been solved.
        kbenson 3y ago
        In the past, "solving" games has often been done through novel changes in hardware and/or approaches and algorithms as I understand it. The person you replied to stated that they didn't know why Othello was an achievement beyond checkers. That doesn't mean it's trivial, but it also doesn't mean it required any additional insight or advances to do so if it's not specifically more complicated than checkers. They then went on to qualify their statement with the information that they don't specifically remember the rules of Othello. In a discussion held in good faith, this should be assumed to be an invitation to add additional insight as to why Othello might be more complicated. You either know the reason why and didn't state it, or believe it's more complicated based on authorities you trust and didn't relate that and instead stated it authoritatively, or don't know it at all and misrepresented it. None of those are very useful for discussion. The former two are easy to rectify by explaining, qualifying, or referring someone to additional info though, and would have been a more useful way to respond. Ultimately though, it sort of feels like you're reaching for an additional interpretation that makes an earlier misinterpretation still valid, when you could just say "oops. Most of my comment still stands as accurate, but I guess I misinterpreted your point." Edit: from looking at you profile, it appears you are a CS professor. Perhaps part of the issue is context. This is a public forum, not a classroom, and while in a classroom your statements can be taken with an assumed authority, nobody necessarily knows that about you here when you post, so some additional qualification of your statements or background may be warranted.
notfed 3y ago
> Trust but verify. Ok, so, do you trust everything posted on the Internet until you verify it? I really think "distrust but verify" is a heck of a better rule, unless your source has a really solid track record.
coldtea 3y ago
>Trust but verify Given that most papers don't replicate, and most hype ends up BS, "don't trust until it's verified" is a much better rule.

throw101010 3y ago

> And yet this is the #1 post on HN right now. Doesn't it make sense to give some visibility to such claims so they can be verified? I would think a lot of people on HN are equipped to take part in this informally. But I agree with you that the title is too categorical and should be updated.

scythe 3y ago

You really wouldn't want to have a post sit on top of Hacker News for an hour and nothing comes of it. Think of how much we could have gotten done on this messageboard otherwise!

mcphage 3y ago
Boy are you gonna feel foolish when the #2 post on HN is “Cancer almost cured”, and you realize how dangerous a diversion this was…

anigbrowl 3y ago

Boo. The paper is cautiously written, addresses a very tractable and well-defined problem, confirms what experts expected, and is straightforward to implement (the source code is on Github). And yet this posturing is the #1 comment on this thread right now.

notfed 3y ago
Great, then let's name it "Othello claimed to be solved", then pound it with peer reviews and reproductions for a bit, then we can phrase it as "Othello is solved".
- SomaticPirate 3y ago
  The source code is on Github? Why not run it yourself and perform the peer review? The implementation is quite straightforward
  - uxp8u61q 3y ago
    You need to prove that the code is correct, that's the tricky part. Otherwise I could just write `return true;` as the whole code and tell people to run it, it's easy, just do the peer review yourself.

cwillu 3y ago

There was absolutely nothing wrong with the superconductor “debacle”. What you're preaching is not skepticism, but cynicism.

coldtea 3y ago
Given the myriad posts and articles about "huge breakthrough", "industry changing" and so on, scepticism in that case would translate to way more cynicism.

g9yuayon 3y ago

I thought the post is #1 on HN because people here are genuinely interested in the topic and would like to at least examine the paper and discuss the merits of it.

klyrs 3y ago

I was watching every development of that superconductivity claim. I was quite enthused! But never once did I move beyond "groundbreaking if true." This doesn't even have the potential to be groundbreaking. Get over yourself, let people get excited about little stuff.

notfed 3y ago
> Get over yourself, let people get excited about little stuff. Some people are looking for facts and science, and some are looking for enthusiasm and excitement. Can't discount either one, I suppose.
- CyberDildonics 3y ago
  Can't discount either one, I suppose. People getting each other excited by pretending fiction is reality is called religion. That's fine if people are honest about it.
- klyrs 3y ago
  You can be enthusiastic and excited while looking for facts and science. In fact, my interest in rooting out truths tends to go hand in hand with my passion for the topic. Negative results are important for scientific advancement, and I was just as excited to see LK99 fully analyzed, despite it not turning out to be the holy grail. We always knew that was a longshot; I enjoyed the journey regardless of the destination. My admonition "get over yourself" is a response to this ridiculous idea that science should be a joyless affair. It isn't, and must not be. Historically, fuddy-duddy downers don't make the big discovaries.
jmye 3y ago
> Get over yourself, let people get excited about little stuff. No one has said you’re not allowed to be excited if you want. Just like GP is allowed to be a wet blanket if they want. Speaking of “get over yourself.” I’m so tired of this “if you don’t completely validate and elevate my feelings then you’re bad/mean and preventing me from feeling what I want” crap. God forbid anyone ever disagree with you.

bigbacaloa 3y ago

[dead]

gfodor 3y ago

Calling it a debacle is an admission you yourself could benefit from your own advice.

satoqz 3y ago

I would not call Preferred Networks an "unknown startup" - They've been prominent within the AI research community for quite some time and have contributed important projects such as CuPy and Chainer, the latter introducing the "define-by-run" concept which was later implemented in frameworks such as PyTorch and Tensorflow.