So in summary it's a hard problem and requires some smarts to get right? Then we agree. Retreating to ASCII is not the answer, and "full text search engines" d

So in summary it's a hard problem and requires some smarts to get right? Then we agree. Retreating to ASCII is not the answer, and "full text search engines" don't even solve the same class of problem, such as say paginating a list of local towns by name.