4 ms·
Even after arguing in favor of XHTML here in another comment, I'd like to make a separate "modest proposal": What if we had a limited kind of markup as part of
by derefr 8y ago
Even after arguing in favor of XHTML here in another comment, I'd like to make a separate "modest proposal":
What if we had a limited kind of markup as part of Unicode, such that all Unicode text could be "marked up", entirely-portably, such that as long as you aren't rewriting the Unicode representation, you're preserving the markup?
Unicode already handles many kinds of markup. We just don't tend to think of these features of Unicode as "markup"—even though many of them have one-to-one equivalents in markup languages!
Most obviously, you've got newlines, the plaintext equivalent of the HTML <br> tag. If Unicode was "just" about representing text, newlines (and horizontal/vertical tabs, non-breaking spaces, zero-width non-joiners, etc.) wouldn't be part of it. These aren't text; they're instructions to a text layout engine, for where to put the text. They're markup, just as much as an HTML <center> tag is!
Less obviously—but more flagrantly—Unicode has the BIDI LTR/RTL instruction codepoints, and support for storing and representing a https://en.wikipedia.org/wiki/Ruby_character https://en.wikipedia.org/wiki/Ruby_character gloss of a given character-sequence embedded alongside said text.
And because of this support, Unicode font rendering engines already have everything they need to be able to render a given "meta-instruction" codepoint that makes a given "run" of characters bold, or italic, or small-caps, or strikethrough; or to indicate that a given run of codepoints should be considered a list item, or even a semantic "paragraph" or "heading."
It's my understanding that the maintainers of the Unicode standard would balk at standardizing this kind of markup within Unicode. But, if all this is possible at the Unicode-parsing level, then why not just create a separate working group who will maintain such markup codepoints as a Unicode Private-Use Area, and then get font-rendering-engine vendors to support said markup codepoints?
---
Side-note: the result would be a lot like creating Unicode-codepoint equivalents for ANSI escape sequences, wouldn't it? In fact, I would imagine that pretty much every common ANSI "Select Graphic Rendition" escape sequence would make sense as a Unicode markup codepoint, save for maybe being translated to prefix-length or per-grapheme-cluster-paired forms rather than being modal switches.
And oddly, I think that such SGR-equivalent codepoints would be within the spirit of Unicode. Unicode wants to allow for the digitization and standards-based preservation of all historical text. Well, how are you supposed to preserve the text-art sent by an 80s BBS, without being able to represent (at least) fg/bg color changes? Unicode already decided to standardize the DOS box-drawing characters for exactly this same reason!
- FigmentEngine 8y agoThats XML?
- cryptonector 8y agoSome of this has been attempted, and deprecated. E.g., there used to be language tags in Unicode corresponding to Internet language tags.
- dec0dedab0de 8y agoThis wont catch on for the same reason noone uses U+001F or U+001E for their tabular data, because its not on the keyboard. Any extra encoding characters would need new keyboards, or maybe touch screens that are good enough to code with. It would be pretty neat though.
- derefr 8y agoYou don't type the BIDI instructions on your keyboard; they're inserted automatically in response to script changes. You don't type Unicode non-breaking spaces with your keyboard—you ask your editor to insert one with an editor-specific command, or you use textual markup that compiles to a Unicode non-breaking space. Etc. Most of the Unicode markup features that already exist—other than newlines—form a backing layer: a standard interchange format for text, when it's not in the internal DOM+ropes representation that most text editors use. It's not a working representation, it's just a representation that text is rendered out to. (In fact, for a lot of these existing markup codepoints, when there's one sitting around "raw" in a text field—rather than having been parsed into a text-editor DOM node—you can't even position the cursor and then delete it. Such codepoints just sort of "stick" to the grapheme-clusters/edit-field boundaries on one or both sides of them, such that deleting the grapheme-cluster will maybe "garbage collect" the associated markup, or maybe leave it there forever. The editor doesn't expect that character to be in its text, breaking the layout. It expects to have parsed it out, and then to put it back in place when it saves the document. The markup codepoints exist to be inline instructions for rendering a run of "baked" text; they don't exist to be a convenient data structure for text-editors to do editing operations against.) So, the idea I'm proposing here, is not that you'd just "be able to type a <b> codepoint"; but rather that: 1. plaintext editors like Notepad and TextEdit—and HTML's plain <textarea> controls—would now have buttons for simple text styling (because now they can!) 2. even OS plaintext edit controls would allow styling of their contents, through an OS-universal styling palette (like macOS's "Fonts panel") that would apply to the cursor/selection of the active window (like it does for macOS rich-text edit controls.) It would essentially be part of the IME, like macOS's minimized-form "Character Viewer" palette is, or like dead-key entry is on X11. 2. WYSIWYG editors for rich-text formats like RTF and Markdown would move to evolved, distinguished storage formats (e.g. ".rtfu" = "text/rtf+umark") where the choice-of-format has the meaning of canonicalizing all styling that is possible using Unicode markup codepoints, as Unicode markup codepoints. 3. The original formats these new formats evolved from ("text/rtf", "text/markdown", etc.) would now be defined specifically as formats that have no knowledge of the meaning of Unicode markup codepoints, such that if a file containing Unicode markup codepoints were to be "Open As"ed the old format, the codepoints would render as �s. (But if you just open the document without an "Open As", the editor would heuristically determine from the fact that the file contains markup codepoints, that the file is actually "text/rtf+umark", and would insist that you "Save As" the file with the new extension.) 4. Each OS's neutral "rich text" drag-object/paste-object format would switch from "text/rtf" to "text/rtf+umark". All text-editing controls on OS apps would be required to both produce this as on drag/copy (even if it's not their storage format), and accept+parse this on drop/paste (even if it's not their storage format.) 5. Each OS's neutral "plaintext" drag-object/paste-object format would become "text/plain+umark"—i.e. plaintext that preserves styling. Thus, copying from a rich-text edit control, to a plaintext edit control, would get you resulting text with editor-specific styling stripped, but retaining the Unicode markup codepoints. 6. If the OS supports arbitrary-Unicode filenames, it should probably accept Unicode filenames with markup codepoints in them, and should probably render that markup when displaying the filename. (Consider: at least in Linux, filenames can contain newlines, and the OS renders those.)