The fact that any of those tasks at all work so well despite tokenization is quite remarkable indeed. You should ask why it is that any of those tasks work, r

The fact that any of those tasks at all work so well despite tokenization is quite remarkable indeed. You should ask why it is that any of those tasks work, rather than ask why counting letter doesn't work. Also, LLMs screw up many of those tasks more than you'd expect. I don't trust LLMs with any kind of numeracy what-so-ever.