Interestingly, I think a strong externalist would argue that a human being raised in a featureless box could not produce text with semantic value to the people

Interestingly, I think a strong externalist would argue that a human being raised in a featureless box could not produce text with semantic value to the people outside the box. One upshot of semantic externalism are brain in vat type arguments, where statements such as “I am perceiving trees” (when they are simulated trees) is false, since the trees that the person is seeing is actually another concept, tree, while tree refers to real world trees. However, tree might be meaningful to the community of people also stuck in the simulation. So it might entail that AI art, in some sense, might be opaque to us but semantically meaningful to other AI raised on the same training data. That would require the AIs to be able to experience aesthetic states to begin with. More precisely, I think it would be akin to the person on the featureless box knowing all the thesaurus concepts to say, pain, but never actually experiencing pain itself. They might be trained to know that pain is associated with certain descriptions such as sharp, unpleasant, dull, heartbreak, and so on, and perhaps extremely complicated and seemingly original descriptions of pain. However, until the human actually qualitatively experiences pain, they only know the corpus of words associated with it. This would be syntactic but not quite semantic. It’s similar to the infamous Mary and the black and white room thought experiment, where even with a complete knowledge of physics, she still learns something new the first time she experiences the blueness of a sky, despite knowing all the propositions related to blue such as that it’s 425nm on the EM spectrum, or that it’s some pattern X of neutrons firing. That said, it’s not clear if this applies to statements other than subjective states. Qualitative descriptions of subjective states like pain, emotions, the general gestalt of the human condition might be empty of content, but perhaps certain scientific and mathematical ones pass the test, as they don’t need to be grounded in direct experience to be meaningful.