I’m not as surprised as many people (I saw a great tweet once that said “Language was the first artificial intelligence, writing was the second. I literally bel

I’m not as surprised as many people (I saw a great tweet once that said “Language was the first artificial intelligence, writing was the second. I literally believe this”, and that’s a broadly accurate description of my worldview too). That said, maybe some of the surprise comes from believing “the map is not the territory” and related ideas? We generally believe that the map is not the territory and this gives us some obviously correct intuitions (like “changing the map doesn’t change the territory”), but maybe it has also given us some subtly incorrect intuitions. I’m not talking about obviously incorrect, like “you can’t understand the territory just by looking at enough maps”. I mean something more subtly wrong. One candidate off the top of my head is an intuition that “maps approximate the territory but necessarily at a lower level of detail (a 1:1 map of the territory would be the same size as the territory), so your understanding of the territory can improve as you read more maps but it can’t improve on the limit of the most detailed map available, because that information literally isn’t there”. I could see that possibly being wrong somehow.