Graphics is different since usually the graphics primitives are the common substrate of all graphics languages, so the interpreter can just be adapted to a diff

Graphics is different since usually the graphics primitives are the common substrate of all graphics languages, so the interpreter can just be adapted to a different backend. For more structured content this is not possible: the natural output of TeX is a sequence of graphics boxes for the glyphs, or something like a DVI format. It is too late to retain semantic informations, e.g. about mathematical formula. Ask anybody which writes LaTeX/HTML converters for example. TeXmacs does a good job to interpret "usual" LaTeX files but you cannot expect to be able to interpret arbitrary files and still retain semantic informations, e.g.: sectioning, references, emphasised texts, theorem statements, change of languages (e.g. from English to C++ to Math...) etc...