I tried it mostly because I was convinced that ChatGPT couldn't've solved this without inner-monologue: it needs to at least be doing something like setting up

I tried it mostly because I was convinced that ChatGPT couldn't've solved this without inner-monologue: it needs to at least be doing something like setting up a frequency table before it could plausibly crack a substitution cipher for real. So I was suspicious there was something amiss here: that you might be hoaxing, or something. I'm almost as surprised as you are that it occasionally guesses Frost and so got it 'right'. There are a lot of famous poems, and I'm a little baffled that your friend would happen to try a poem that it would successfully guess on the first few tries. Either there's some deeper connection between Frost & substitution ciphers than I know of (maybe a tutorial somewhere?), or people are engaged in a lot of ChatGPT interactions such that even extremely improbable guesses like this will still occasionally succeed.