3 ms·
When we cast our randomness as a string we impose a lot of structure. Proportionally, getting to English from there might involve a lot less new structure than
by ganzuul 7y ago
When we cast our randomness as a string we impose a lot of structure. Proportionally, getting to English from there might involve a lot less new structure than in the previous step.
Often I wonder if I'm just spouting gibberish, saying things like this, but it does make sense to me.
- yters 7y agoNo, the information cost grows exponentially going from random letters to English sentences. We can measure this with crossentropy to get a probability of generating a well formed English sentence from a uniform distribution over the letters. To generate just a few hundred words we end up needing more trials than are available in the lifespan of our universe.