Letter questions are atypically hard for GPT2/3, FWIW. The input is dictionary compressed data and it only learns stuff about letters indirectly. As a result

Letter questions are atypically hard for GPT2/3, FWIW. The input is dictionary compressed data and it only learns stuff about letters indirectly. As a result GPT2 is hopeless: it does however correctly answer color that's a fruit-- an answer which wasn't obvious to me. But you can't generally get it to operate on first letters. GPT3 has shown some capability to operate on first letters, but it's less good than it is at other things. GPT2/3 also have fixed depth computation, so compound requests can be hard for it (though GPT3 seems a lot better)