For some reason, the Go example reminds me of the monolith in Clarke's 3001 and the protomolecule in The Expanse. The book authors describe both as very advance

For some reason, the Go example reminds me of the monolith in Clarke's 3001 and the protomolecule in The Expanse. The book authors describe both as very advanced technology that lack consciousness. They do what they're designed to do, but they do not experience doing it, and thus lack the advantage of a certain kind of relfective awareness the conscious constructs they create, such as Dave or Miller, do possess. The Star Trek holodeck also sometimes accomplishes this. I have no idea whether it's possible to create a machine that can generate conscious simulations without itself being conscious. But there's no Bayesian reasoning to apply to such scenarios, since we have no idea as of yet. They're philosophical arguments. And applying to that to current technology is misplaced.