I can see the theoretical advantages of such a concept, but I think a key limitation is that we don't have appropriate amounts of data with accurate phonetic re

I can see the theoretical advantages of such a concept, but I think a key limitation is that we don't have appropriate amounts of data with accurate phonetic representation. The potential advantage of using a phonetic representation is that it can have different relevant information than written spelling does. However, if you take the written spelling and pass it through some rules that transform it to what the phonetic representation might be... that transformation can only destroy information, not add it; you'd just be better off using the source data directly. Now if at some point we get to a place where most of the training data is audio (i.e. the quantity of spoken words in available audio data becomes larger than current written data on internet and in libraries), then phonetic representation would make all sense, being closer to the source data. But if we're talking about purely tokenization - I think your suggestion is effectively halfway towards morphologically based tokenization, splitting into morphemes (which tend to map to semantics), and that is getting explored. The problem is, for an apples-to-apples comparison you need equally sized models and changes to tokenization require a complete retraining of the model; so doing a comparison on GPT-3 or GPT-4 scale is very expensive (too expensive for "would be interesting" to justify it), and measuring the effect on small models won't necessarily be very indicative of how it will affect large models.