The only experiment I know of and would consider a serious attempt at reproduction would be Tuxedage's series, https://www.lesswrong.com/posts/FmxhoWxvBqSxhFeJ

The only experiment I know of and would consider a serious attempt at reproduction would be Tuxedage's series, https://www.lesswrong.com/posts/FmxhoWxvBqSxhFeJn/i-attempted-the-ai-box-experiment-and-lost https://www.lesswrong.com/posts/FmxhoWxvBqSxhFeJn/i-attempte... https://www.lesswrong.com/posts/dop3rLwFhW5gtpEgz/i-attempted-the-ai-box-experiment-again-and-won-twice https://www.lesswrong.com/posts/dop3rLwFhW5gtpEgz/i-attempte... https://www.lesswrong.com/posts/oexwJBd3zAjw9Cru8/i-played-the-ai-box-experiment-again-and-lost-both-games https://www.lesswrong.com/posts/oexwJBd3zAjw9Cru8/i-played-t... His total is 3 for 3. I do not know how to explain these results without either taking them to be honest attempts at a fair experiment or by assuming those involved colluded. I find the latter absurd, given my priors about the honesty of members of LessWrong (Yudkoswky in particular, though he wasn't involved in the reproduction). > If I told you I reenacted the experiment at home with a friend, would you consider this a reproduction? Someone saying they reproduced it on the internet would convince you? It is not so simple. I would want evidence that you and your friend were smart and had a decent understanding of the domain, and that your friend was in a similar state of unbelief about the plausibility of being convinced. I would want a statement that it was a serious attempt at doing things "the hard way" and true to the experiment, on both sides, lest you get [1]. Of course I would want the standard rules, or a reasonable modification publicly stated, in addition. [1] https://pastebin.com/Jee2P6BD https://pastebin.com/Jee2P6BD > What is the goal of the experiment? To show that "I can't imagine anything that even a transhuman could say to me which would change [my mind]" is not evidence, and should not be treated as such. To provide evidence that "humans are not secure systems". You say "very low stakes", but Yudkowsky convinced someone who had offered a $5000 handicap. That hardly seems like a trivial quantity. > [maybe it's all a lie] You have to be very cynical to take this worldview. > we don't even know what a "transhuman" mind would be like The experiment is under assumption of a true singularity, ergo. nigh-unlimited intelligence. I can discuss what outcomes I think are likely for AI development, or which are merely plausible, but the experiment is about one particular hypothetical, so that would be a different conversation. > the participants seem to be selected from a pool of people fascinated by this kind of questions and who would be open to suggestion that a "transhuman" mind can convince them of stuff I am unconvinced that this experiment would work if the gatekeepers did not have an understanding of the topic; they are meant to play a gatekeeper, after all. A person who considers the singularity plausible but thinks an AI box is effective seems like the perfect control should the singularity happen and people want to figure out whether to AI box it.