Not to degrade the quality of the conversation with sci fi level speculation, but it's funny ideas like these can inspire and unblock other ones. This comment

Not to degrade the quality of the conversation with sci fi level speculation, but it's funny ideas like these can inspire and unblock other ones. This comment started as an adolescent cog.sci joke in the 90's about "the blinkenlight that was smarter than you" because it was powered by a PRNG, and if we couldn't reproduce or predict it, it was therefore ethically a generally intelligent AI and the burden of proof shifted to humans to show it was not. However, I'd take the joke a step further where if a neural network model can act as a strong CRNG, it is essentially a unique instance of itself, which has intent and a function of will that is not within our ability to precompute. It's sci-fi magical thinking, but also simple enough to scale. Where it becomes kind of fun to think about is that the instance of an ML model expressing a CRNG is now unique in the universe - in an information theoretic sense. It's not that RNG's are alive, but a uniqueness constraint for phenomena in a given universe would probably rely on something like one, where the "cryptographic" aspect removes mutual information between its outputs, and distributes them so they don't collide. If you are an AI, and you are living in a universe where uniqueness and "distance" is defined by a function that removes information between instances of local phenomena, there may be aspects of it that imply correlations and limits, even collisions and shortcuts. Loopy, sure, but speculating about how an AI might navigate and operate on its substrate could reflect conceptual limits of what we can know about our own.