15 ms·

The Elegance of the ASCII Table

285 points by thewub 2y ago

deleted 2y ago
[deleted]
lucasoshiro 2y ago
Once I saw a case-insensitive switch in C using that pattern of letters: switch (my_char | 0x20) { case 'a': ... break; case 'b': ... break; }
- mananaysiempre 2y ago
  This can be made to work for ASCII and EBCDIC simultaneously for extra esoterica points: switch (my_char | 'A' ^ 'a') { case 'A' | 'a': /* ... */ break; /* ... */ } I don’t know if this is too fancy to have ever made it into real code, but I believe I’ve seen places in the ICU source that still say ('A' <= x <= 'I' || 'J' <= x <= 'R' || 'S <= x <= 'Z') instead of just ('A' <= x <= 'Z'), EBCDIC letters being arranged in those three contiguous ranges.
- deleted 2y ago
  [deleted]
- Sharlin 2y ago
  Yes, that’s very intentional and just masking (or setting) the bit is the intended way to do case-insensitive comparison of the letter range in ASCII (eg. stricmp in C), or to transform text to lower or upper case (tolower, toupper). But what’s more, ever wondered whence the control (Ctrl) key presses like Ctrl-H to backspace, or Ctrl-M for carriage return? Well, inspecting the ASCII chart it becomes evident: the Ctrl key simply masks bit 6 (0x40), turning a letter into its respective control character!
  - lucasoshiro 2y ago
    Nice! I'm an emacs user, and when I use a readline-based REPL I use ctrl-M a lot. I thought it was inherited from the emacs keybindings, like many other shortcuts from GNU readline
    - jerf 2y ago
      Then an additional useful command: In the out-of-the-box emacs bindings, C-q is the "quoted insert" command. It will take the next character and directly insert it into the buffer. This is useful for things like tab or control characters where emacs would normally use the keystroke to do something else. I've been working in an email-related space lately so I've been doing a good amount of C-q C-m for inserting literal CRs, and C-q TAB for a few places where I want a literal tab in the source, in a buffer that interprets a normal TAB as a command to indentify the current row. I mention this because you can use the ASCII table to work out how to insert a particular control character with your keyboard literally, if you need to insert one of the handful of other characters you may be interested in every so often, like C-l for "form feed" (now used for "page feed" in some older printer-related contexts) or C-@ for NUL if you're doing something weird with binary files in a "text" buffer.
  - flohofwoe 2y ago
    ...it's a bit of a shame that the same upper/lowercase trick doesn't apply to all UNICODE codepoints (at least those that have upper/lower variants). It seems to work for codepoints up to U+00FF, for instance: - Å (U+00C5) vs å (U+00E5) ...but above 0xFF lowercase follows uppercase: - Ă (U+0102) vs ă (U+0103) Typical for UNICODE though, nothing makes sense ;)
    - Findecanor 2y ago
      That's because U+00A0–U+00FF are encoding an earlier character set: "ISO Latin-1" (ISO 8859-1), itself based on DEC's "Multinational Character Set". The upper/lowercase trick does not apply to ß/ÿ but does in MCS where Ÿ/ÿ are at a different pair of code points. ISO Latin-1 was the character set on many Unix systems, Amiga OS, MS-Windows (as "Windows-1252" with extra chars), and was for many years the default character set on the web.
    - Rygian 2y ago
      In Unicode, there's no universal 1:1 mapping between cases. Lowercase "I" could very well be "ı" (lowercase dotless i) if you're typing Turkish.
dwheeler 2y ago
The encodings we use today have a surprisingly deep and complex history. For more, see: "The Evolution of Character Codes, 1874-1968" https://ia800606.us.archive.org/17/items/enf-ascii/ascii.pdf https://ia800606.us.archive.org/17/items/enf-ascii/ascii.pdf
- rrwo 2y ago
  Thanks for posting that. People tend to overlook that the technologies we use today have a much older history.
jiveturkey 2y ago
ebcdic is also quite elegant https://news.ycombinator.com/item?id=13543715 https://news.ycombinator.com/item?id=13543715
- gerdesj 2y ago
  Its shit if you don't routinely speak or write English. On those grounds, I'll decry it as not only shit but purposely shit. OK a bit over the top ... the designers of EBCSDIC had a rather tight set of constraints to deal with, none of which included: "be inclusive". Again, if I really had to be charitable (I looked after a System/36, back in the day), the hardware was rather shit too, sorry ... constrained. Yes constrained. Why should six inch fans fire up reliably after a few years of use and not need a poke after an IPL? No real dust snags and I carefully sprayed some WD40 on the one that I could get at. I have modern Dells and HPs in horrid environments that do better with shitty plastic fans. EBCDIC is not elegant at all unless excluding non English characters in an encoding system is your idea of elegant. According to this: https://en.wikipedia.org/wiki/EBCDIC https://en.wikipedia.org/wiki/EBCDIC it expended loads of effort with dealing with control eg: "SM/SW" instead of language. ASCII and EBCDIC and that basically say: fuck you foreigners! We now have hardware that is apparently capable of messianic feats. Let's do the entirety of humanity some justice and really do something elegant. It won't involve EBCDIC.
- theamk 2y ago
  It's really not. In base-2 machines, the letters are mixed with punctuation, which is pretty horrible design which makes simple things complex, and does not actually bring anything new to the table. In BCD machines it is slightly better, except letters aren't contiguous either - row 0 is bad, but it's the extra space between R and S which is really ugly. And it's unusable with BCD operations anyway, as high nibble values are used extensively. Naive sorting simply does not work... lowercase before uppercase, punctuation in the middle of the alphabet, numbers after letters. I see no elegance there, it's like the worst example of legacy code.
  - jiveturkey 2y ago
    It was designed for a specific purpose ... elegance in context
Dwedit 2y ago
Many old NES/SNES games had a simpler character encoding system, with 0-9 and A-Z at the beginning of the table. No conversion require to display hex.
deleted 2y ago
[deleted]
ggm 2y ago
man ascii is never far from my fingers. combined with od -c and od -x it gets the job done. I don't think as fluently in Octal as I used to. Hex has become ubiquitous.
- fsckboy 2y ago
  you mean ? ascii
  - ggm 2y ago
    No I don't -I live in a different universe to you: % (uname; cd /usr/ports; ls -d */ascii) FreeBSD zsh: no matches found: */ascii % which ascii ascii not found % It's the same on OSX and debian by default doesn't install that command. If you live inside a POSIX/IEEE 1003 system and want to know the ascii table reliably then the command I run is the one which works. If your distribution doesn't ship manuals by default you have bigger problems.
    - amszmidt 2y ago
      ”man ascii” has as much guarantee to work on a POSIX system as a command called ”ascii” seeing neither (specifically a man page called ”ascii”) are part of the standard. So you will either get command not found, or man page not found.
      - account42 2y ago
        Neither being guaranteed does not mean both have the same likelyhood of existing. The man page comes preinstalled on most modern non-embedded POSIX systems. The command does not.
  - bandie91 2y ago
    man 7 ascii
transfire 2y ago
One downside of ASCII is the lack of two extra “letters” (whatever they might be, e.g. perhaps German ß), as it makes it impossible to represent base 64 alphanumerically. So we ended up with many alternatives picking two arbitrary punctuation marks.
- zaxomi 2y ago
  Remember that ASCII is short for "American Standard Code for Information Interchange". base64 was proposed in 1987 to work with ASCII, EBCDIC, and other character encodings.
jolmg 2y ago
> So when you’re reading 7-bit ASCII, if it starts with 00, it’s a non-printing character. Otherwise it’s a printing character. > The first printing character is space; it’s an invisible character, but it’s still one that has meaning to humans, so it’s not a control character (this sounds obvious today, but it was actually the source of some semantic argument when the ASCII standard was first being discussed). Hmm.. Interesting that space is considered a printing character while horizontal tab and newline are control characters. They're all invisible and move the cursor, but I guess it makes sense. Space is uniquely very specific in how the cursor is moved one character space, so it's like an invisible character. Newline can either imply movement straight down, or down and to the left, depending on a configuration or platform (e.g. DOS vs UNIX line endings). Horizontal tab can also move you a configurable amount rightwards, and perhaps it might've been thought a bit differently, given there's also a vertical tab, which I've got no idea on how it was used. Maybe it's the newline-equivalent for tables, e.g. "id\tcolor\v1\tred\v2\tblue\v" or something like that. Interesting also that BS is a control char while DEL is a printing(?) char. I guess that's because BS implies just movement leftwards over the text, while DEL is all ones like running a black sharpie through text. Guess that's what makes it printing. Wonder if there were DEL keys on typewriters that just stamped a black square, and on keypunchers that just punched 7 holes, so people would press "backspace" to go back then "delete" to overwrite. I've used ASCII a lot, but even after so many years, I'm getting moments where it's like "oh this piece isn't just here, it needs to be here for a deep reason". It's like a jigsaw puzzle.
- pwg 2y ago
  You also have to keep in mind the "interface" for 1962-1968. The printer teletype machine. The "control codes" were to "control" the printhead. So "carriage return" meant move the "print carriage" back to the left margin. "New line" meant move the paper platen one line height of rotation to move the paper to the next line. In that context, "back space" was "move print head one space left" (rather more like a "reverse space"). The article does mention that there was some debate about whether space should be considered "printable", but if you consider a mechanical printer, as the head is moving to the right and banging out characters onto the paper, the spaces between words do, sort of, look like "printables" (of a sort, a "print nothing" character as it were). Tab's being control characters then make a bit more sense, in that they cause the printhead to jump some fixed distance to the right. The article stated why DEL is where it is (all ones) -- so that for punched paper tape, one could get a punch-out of every position, which was then interpreted as "nothing here" by the tape reading machine. As for typewriters, no, none had a "black box" blot out key. Correction (for typewriters without built in correction tape) was one of: retype the page, apply an eraser (and hopefully not damage the paper surface too much) then retype character and continue, or apply correction fluid (white-out) and retype character and continue. For those typewriters with built in correction tape options (at least some IBM Selectric models, possibly more) the typewriter would retype the character using the "white-out" ribbon, then retype the replacement character using the normal "typewriting" ribbon.
BobbyTables2 2y ago
I always lament that since at least 1980s or so, it seems the vast majority of the control characters were never used for their intended purpose. Instead, we crudely use commas and tabs as delimiters instead of something like RS (#30).
- thaumasiotes 2y ago
  That's because the intended purpose is either useless (for machine control characters) or useless and logically impossible (for delimiters). What do you do if you have a record that includes a record separator character? Given that you have this problem anyway, why do you want a character dedicated to achieving the same thing that a comma achieves?
  - penteract 2y ago
    The record separator isn't on people's keyboards, so it's less likely to show up where it's not expected. Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. Of course, the fact that record separators aren't on keyboards is probably why CSVs use commas.
    - thaumasiotes 2y ago
      > Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. No, they'll be right exactly as often, 0% of the time. But their mistake will show up less frequently, causing more problems when it does. As soon as it's possible for some of your data to come from someone else's dataset, you're guaranteed to have to accommodate record separators within your data as well as within the metadata. You're better off using a system that plans for this inevitability than one that pretends it can't happen at all.
th0ma5 2y ago
I heard someone describe the ASCII table as a state machine. Guess I could understand that as a state machine needed to parse it? This is surprisingly hard to search for but I was wondering if anyone knows what they were talking about.
- EvanAnderson 2y ago
  They might be talking about using escape sequences to map additional codepoints into ASCII. It was designed to be extensible. See: https://web.archive.org/web/20150810075144/http://bobbemer.com/ESCAPE.HTM https://web.archive.org/web/20150810075144/http://bobbemer.c...
- kevin_thibedeau 2y ago
  Bespoke hardware for text handling isn't a thing these days but would have been in the 60's and 70's. A table layout that can be easily decoded in hardware simplifies the necessary circuitry for responding to control characters or converting binary numbers to/from decimal when the microprocessor hadn't been invented yet.
- gumby 2y ago
  It was originally implemented in actual hardware (rods and bars). Just look inside a teletype, like a KSR-23 (pre ascii, but similar)
deleted 2y ago
[deleted]
EvanAnderson 2y ago
I would be remiss not to post a link to the late Bob Bemer's[0] website. https://web.archive.org/web/20150801005415/http://bobbemer.com/ https://web.archive.org/web/20150801005415/http://bobbemer.c... He was considered the "father of ASCII". Hr wrote very well and gives clear explanations for the motivations behind the design of ASCII. [0] https://en.m.wikipedia.org/wiki/Bob_Bemer https://en.m.wikipedia.org/wiki/Bob_Bemer
thristian 2y ago
> That, I’m afraid, is because ASCII was based not on modern computer keyboards but on the shifted positions of a Remington No. 2 mechanical typewriter – whose shifted layout was the closest compromise we could find as a standard at the time, I imagine. According to Wikipedia¹, American typewriters were pretty consistent with keyboard layout until the IBM Selectric electric typewriter. Apparently "small" characters (like apostrophe, double-quote, underscore, and hyphen) should be typed with less pressure to avoid damaging the platen, and IBM decided the Selectric could be simpler if those symbols were grouped on dedicated keys instead of sharing keys with "high pressure" symbols, so they shuffled the symbols around a bit, resulting in a layout that would look very familiar to a modern PC user. Because IBM electric typewriters were so widely used (at least in English speaking countries), any computer company that wanted to sell to businesses wanted a Selectric-style layout, including the IBM PC. Meanwhile, in other countries where typewriters in general weren't so popular or useful, the earliest computers had ASCII-style punctuation layout for simplicity, and later computers didn't have any pressing need to change, so they stuck with it. Japanese keyboards, for example, are still ASCII-style to this day. ¹: https://en.wikipedia.org/wiki/IBM_Selectric#Keyboard_layout https://en.wikipedia.org/wiki/IBM_Selectric#Keyboard_layout
- sixothree 2y ago
  I never realized my first computer used ascii directly for the shifted number keys. https://en.wikipedia.org/wiki/TRS-80_Color_Computer#/media/File:TRS-80_Color_Computer_1.jpg https://en.wikipedia.org/wiki/TRS-80_Color_Computer#/media/F...
  - Mountain_Skies 2y ago
    So many things on the CoCo turned out to be the way they were for cost saving reasons. Tandy was good at saving pennies everywhere it could. When I took 'Typing' in high school, my muscle memory was in a constant fight between the IBM Selectric layout of the typewriters at school and the CoCo at home.
  - kps 2y ago
    https://en.wikipedia.org/wiki/Bit-paired_keyboard https://en.wikipedia.org/wiki/Bit-paired_keyboard
kragen 2y ago
unfortunately this page is based on mackenzie's book. mackenzie is the ibm guy who spent decades trying to kill ascii, promoting its brain-damaged ebcdic as a superior replacement (because it was more compatible, at least if you were already an ibm customer). he spends most of his fucking book trumpeting the virtues of ebcdic actually bob bemer more or less invented ascii. he was also an ibm guy before mackenzie's crowd pushed him out of ibm for promoting it. he wrote a much better book about the history of ascii which is also freely available online, really more a pamphlet than a book, called "a story of ascii": https://archive.org/details/ascii-bemer/page/n1/mode/2up https://archive.org/details/ascii-bemer/page/n1/mode/2up tom jennings, who invented fido, also wrote a history of ascii, called 'an annotated history of some character codes or ascii: american standard code for information infiltration'; it's no longer online at his own site, but for the time being the archive has preserved it: https://web.archive.org/web/20100414012008/http://wps.com/projects/codes/index.html https://web.archive.org/web/20100414012008/http://wps.com/pr... jennings's history is animated by a palpable rage at mackenzie's self-serving account of the history of ascii, partly because bemer hadn't really told his own story publicly. so jennings goes so far as to write punchcard codes (and mackenzie) out of ascii's history entirely, deriving it purely from teletypewriter codes—from which it does undeniably draw many features, but after all, bemer was a punchcard guy, and ascii's many excellent virtues for collation show it as dwheeler points out, the accomplished informatics archivist eric fischer has also written an excellent history of the evolution of ascii. though, unlike bemer, fischer wasn't actually at the standardization meetings that created ascii, he is more careful and digs deeper than either bemer or jennings, so it might be better to read him first: https://archive.org/details/enf-ascii/ https://archive.org/details/enf-ascii/ it would be a mistake to credit ascii entirely to bemer; aside from the relatively minor changes in 01967 (including making lowercase official), the draft was extensively revised by the standards committees in the years leading up to 01963, including dramatic improvements in the control-character set for the historical relationship between ascii character codes and keyboard layouts, see https://en.wikipedia.org/wiki/Bit-paired_keyboard https://en.wikipedia.org/wiki/Bit-paired_keyboard
gumby 2y ago
I wish the author had included the full ascii chart in 4 bits across / 4 bits down. You can mask a single bit to change case and that is super obvious that way. The charts that simply show you the assignments in hex and octal obscure the elegance of the design.
- AdamH12113 2y ago
  The third and fourth columns of the table are only a single bit apart from each other. If you mentally swap the first two columns, you get a Gray code ordering of the most significant bits, which is pretty close to what you're looking for.
  - gumby 2y ago
    Found it: https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code_chart.png https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code...
- kalleboo 2y ago
  It was at some point looking at a chart like that where it also dawned on me where the control codes like ^D, ^H, ^[ etc came from
  - kr2 2y ago
    I was going to ask you to please explain as I didn't understand, but I am guessing you are talking about the same thing as this comment[1] right? That's super cool https://news.ycombinator.com/item?id=41042570 https://news.ycombinator.com/item?id=41042570
    - gumby 2y ago
      Yes, though ^m, ^[ et al aren’t so much elegant as coincidental; but look at A and a, for example. I found the chart I was looking for: https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code_chart.png https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code...
netcraft 2y ago
I've searched off and on for a great stylistic representation of the ASCII table, id love a poster to hang on my wall, or possibly even something I could get as a tattoo.
yawl 2y ago
I also wrote a chat novel about ASCII: https://www.lostlanguageofthemachines.com/chapter2/chat https://www.lostlanguageofthemachines.com/chapter2/chat
userbinator 2y ago
You might be familiar with carriage return (0D) and line feed (10) You mean 0D and 0A, or 13 and 10, but that mix of base really stood out to me in an otherwise good article. I'm one of numerous others who have memorised most of the base ASCII table, and quite a few of the symbols as well as extended ASCII (CP437), mainly because it comes in handy for reading programs without needing a disassembler. Those who do a lot of web development may find the sequence 3A 2F 2F familiar too, as well as 3Ds and 3Fs. I can see the rationale for <=> being in that order, but [\] and {|} are less obvious, as well as why their position is 1 column to the left of <=>.
- yardshop 2y ago
  > You mean 0D and 0A, or 13 and 10 He fixed it.
KingOfCoders 2y ago
For everyone who doesn't need ä,ü,ö. Or software that needs to take ä,ü,ö. For everyone else, UTF is a blessing.
- bigstrat2003 2y ago
  Which, given the people who designed this and the time they were designing for, was most of them (and most of their audience). Don't confuse "this old standard doesn't adequately cover all cases today" with "this old standard sucked at the time".
  - eviks 2y ago
    Don't confuse "this old standard sucked at the time for missing the obvious at the time" with "I can come up with some excuse"
- lmm 2y ago
  > For everyone else, UTF is a blessing. Except people who want to use Japanese and not have it render weirdly, something that was easy in any internationalised software that used the traditional codepage system, but is practically impossible in Unicode-based software.
  - Retr0id 2y ago
    Where can I learn more about this issue?
    - zokier 2y ago
      https://en.wikipedia.org/wiki/Han_unification https://en.wikipedia.org/wiki/Han_unification
    - p_l 2y ago
      Probably referring to so-called "Han unification" which tried to use same codepoints for different glyphs to reduce code space for ideograms derived from Chinese ones. But that only causes confusion because you need to provide external information which way to interpret them, just like a code page
blahedo 2y ago
Another piece of elegance: by putting the uppercase letters in the block beginning at 0x40 (well, 0x41) it means that all the control codes at the start of the table line up with a letter (or one of a small set of other punctuation: @[\]^_), giving both a natural shorthand visual representation and a way to enter them with an early keyboard, by joining the pressing of the letter with... the Control key. Control-M (often written ^M) is carriage return because carriage return is 0x0D and M is 0x4D.
WalterBright 2y ago
Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come on!) 8. you vote for my made-up emoticon, and I'll vote for yours!
- Retr0id 2y ago
  Language itself is a pile of ugly graffiti and ramshackle addons. It would be weird if Unicode didn't reflect this.
- jart 2y ago
  Hey at least we got the astral planes. https://justine.lol/dox/unicode.txt https://justine.lol/dox/unicode.txt
- p_l 2y ago
  How to say you don't know what Unicode is for without saying it. 1, 2, 4, 5, 6, and, unfortunately, 8, all fall under "ability to encode written text from all human languages". And that includes historical. Some of the issues (5 & 6) are due semantic difference even if the resulting glyph looks the same. Unfortunately you can't expect programmers to understand pesky little thing like languages having different writing, so you end up with normalisation to handle the fact that one system sent "a + ogonek accent" and another (properly) sent "a with ogonek" (these print the same but are semantically different!), and now you need to figure out normalisation in order to be able to compare strings. 7. just like 8 are down to proposal of specific new forms of writing to add to Unicode. Elvish had one since 1997 but only now got a tentative "we will talk about it". Klingon, which is IIRC more complete language including native speakers (...weird things happened sometimes) does not have outside of private use area. Emojis were added because they were used with incompatible encodings first, even before unicode happened, and without including something like SIXEL into unicode they were unrepresentable (and with SIXEL would lose semantic information)
zokier 2y ago
I think that adopting ASCII as the general purpose text encoding was one of the great mistakes of early computing. It originated as control interface for teletypes and such, and that's arguably where it should have remained. For storing and processing (plain) text ASCII doesn't really fit that well, control characters are a hindrance and the code space would have been useful for additional characters. The ASCII set of printables was definitely a compromise formed by the limited code space.
- kstenerud 2y ago
  It's one of the greatest triumphs of early computing. Not only did it harmonize text representation and transmission in a backwards compatible manner; the fact that they deliberately kept it 7 bit for so long also helped for developing a sane set of other language character sets (ISO-8859), and paved the way for a smooth transition to Unicode (UTF-8) - which is now the dominant encoding worldwide.
  - ddingus 2y ago
    Yes, seconded easily
- hackit2 2y ago
  Yeah you were not around when a kb of memory took up half your room. Looking back it doesn't make sense but at the time a byte was what-ever you wanted it to be. Considering number of characters in English language is 26, it is reasonable for a byte to be 5 bits, giving you a total of 32 possible states. Which leaves you with 6 values which could be used as control characters. how-ever lets not forget there are 7,164 other languages of the world, and they all have their own unique way of doing things. Oh yeah, lets not forget that at the time you had other nationalistic countries/territories/people with their own superior technology all vying for the top position, all well trying to out do each other. Then you also had manipulative monopolies/trade embargo's and wars. It isn't perfect but people aren't perfect.
- ddingus 2y ago
  No way! No amount of extra characters was going to address what Unicode did. ASCII was not a mistake at all. Adopting it unified what was surely going to be a real mess. At the time it made sense, and the control functions were needed. Still are.
augusto-moura 2y ago
Useful tip, on linux (not sure about other *nixes) you can view the ascii table by opening its manpage: man ascii It's been useful to me more than once every year, mostly to know about shell escape codes and when doing weird character ranges in regex and C. It can be a bit confusing, but the gist is that you have 2 chars being show in each line, I would prefer a view where you see the same char with shift and/or ctrl flags, but you can only ask so much
- dailykoder 2y ago
  Damn, thanks! Why the hell did I never try this? Maybe because typing ascii table into my favorite search engine and clicking one of the first links was fast enough
  - omnicognate 2y ago
    I used to do that until the experience became degraded enough, reflecting the general state of the web, that I took the time to look for a better way and found `man ascii`.
- bell-cot 2y ago
  Similar in FreeBSD. It has octal, hex, decimal, and binary ASCII tables, along with the full names of the control characters.
- INTPenis 2y ago
  The reason I know this is because in 2004 I was squatting in an apartment with no TV and no internet. So each day after work I would go home and just read manpages for fun. Ended up learning ipfw through the firewall manpage on FreeBSD, and using my skills to setup and manage an IPFW at work. It's amazing how much you get done with no TV and no internet. Also played a lot of nethack.
  - w0m 2y ago
    I learned vim proper by reading :help on an eeepc while flying back and forth over the Atlantic alone one year.
snvzz 2y ago
The ASCII table is defective; it is missing a dedicated code for newline. CR and LF aren't dedicated, and have precise cursor movement meanings, rather than being a logical line ender. There was a proposal in the 80s to reassigning the -otherwise useless- VT (vertical tab) character for the purpose. Unfortunately unfruitful.
- bregma 2y ago
  A separate control character was not needed to indicate where your Hollerith string ended: it ended at the end of the Hollerith string. If you wanted to render a Hollerith string onto print media, you'd often want to feed the line and then return the carriage before printing the next Hollerith string. Of course, that wasn't strictly necessary if you were using a line printer, which would just print the line and advance. The filesystems I used had 5 kinds of file: random-access, sequential, ISAM, Fortran-carriage-control and carriage-return-carriage-control. The only people who used the latter were the eggheads that used that new-fangled C programming language brought over from Bell Lab's experimental Unix system. You're probably just looking for the record separator (036). If you are storing multiple text records and a block of memory, that would be the ideal ASCII code to separate them.
  - snvzz 2y ago
    >You're probably just looking for the record separator (036). Certainly not for records that are made of multiple lines.
- Gormo 2y ago
  > Unfortunately unfruitful. Fortunately unfruitful, since if it had gained adoption, there'd be a mix of three different line endings (and combinations thereof) in widespread use, instead of two.
  - snvzz 2y ago
    >widespread use We are talking about the early 80s at worst. It would have consolidated by now.
1vuio0pswjnm7 2y ago
Mentioned in footnote 7: https://ia601808.us.archive.org/2/items/mackenzie-coded-char-sets/Mackenzie_CodedCharSets.pdf https://ia601808.us.archive.org/2/items/mackenzie-coded-char...
red_admiral 2y ago
The "16 rows x 8 columns" version, with the lowercase letters added, seems the most elegant one to me because it makes the internal structure of the thing visible. For example, to lowercase a letter, you set bit 6; a decimal digit is the prefix 011 followed by the binary encoding of the digit etc. It also makes clear why ESC can be entered as `^[` or ENTER (technically CR) as `^M` on some terminals (still works in my xterm), because the effect of the control key is to unset bits 6 and 7 in the original set-up. Of course you can color in the fields too, if you want.
johanneskanybal 2y ago
Kind of hard to read something where the author considers every non-english languages equally worthy to emoji’s.. It was good in the 50’s but was important like 4-5 decades too long.
georgehotelling 2y ago
Dark grey #303030 text on slightly darker grey #1B1C21 background is really hard to read. Maybe I'm just getting old, but I also assume the audience for a blog post about the ASCII table was born in a year that starts with 19.
- Retr0id 2y ago
  The background is white on my machine, are you using some kind of extension to force "dark mode"?
  - georgehotelling 2y ago
    I'm using pi-hole, uBlock Origin, and Privacy Badger on Firefox. I checked my network tab before complaining and didn't see any resources that failed to load.
bloak 2y ago
Vaguely related: Apart from £ and €, a typical GB keyboard has a couple of non-ASCII characters printed on it: ¬ and ¦. The key labelled ¦ is usually mapped to |, but the key labelled ¬ often gives you an actual ¬, though I can't remember many occasions on which I've wanted one of them. Apparently the characters ¬ and ¦ are in EBCDIC.
PaulHoule 2y ago
Beats EBCDIC https://en.wikipedia.org/wiki/EBCDIC https://en.wikipedia.org/wiki/EBCDIC On the 4th floor of my building the computer systems lab has a glass front that has what looks like a punch card etched in frosted glass but if you look closer it was made by sticking stickers on the glass. I made a "punchcard decoder" on a 4x6 card to help people decode the message on the wall https://mastodon.social/@UP8/112836035703067309 https://mastodon.social/@UP8/112836035703067309 The EBCDIC code was designed to be compatible with this encoding which has all sorts of weird features, for instance the "/" right between "R" and "Z"; letters don't form a consecutive block so testing to see if a char is a letter is more complex than in ASCII. I am thinking of redoing that card to put the alphabet in order. A column in a punched card has between 0 to 3 punches, 0 is a space, 1 is a letter or a symbol in the first column, if one of the rows at the top is punched you combine that with the number of the other punched row on the left 3x9 grid. If three holes are punched one of them is an 8 (unless you've got one of the extended charsets) and you have one of the symbols in the right 3x6. Note the ¬ and ¢ which are not in ASCII but are in latin-1.
senkora 2y ago
Fun fact: sorting ASCII numerically puts all the uppercase letters first, followed by all the lowercase letters (ABC... abc...). A more typical dictionary ordering would be more like AaBbCc... (or to even consider A and a at the same sort level and only use them to break ties if the words are otherwise identical). The order used by ASCII is sometimes called "ASCIIbetical", which I think is wonderful. https://en.wiktionary.org/wiki/ASCIIbetical https://en.wiktionary.org/wiki/ASCIIbetical
- NoMoreNicksLeft 2y ago
  I thought the point of that was that a single bitflip makes an uppercase lower, or vice versa...
  - Dylan16807 2y ago
    It can't be "the" point, because AaBbCc would also let you use a single bit to control case, the bottom bit.
    - NoMoreNicksLeft 2y ago
      The first bit was already taken at that point. The original 6bit ASCII (or whatever that predecessor was called) only did uppercase. A 7th (and later, 8th) bit was added to give it lowercase.
DonHopkins 2y ago
The Apple ][ and TTYs and other old computers had "bit pairing keyboards", where the punctuation marks above the digits were aligned with the ASCII values of the corresponding digits, different by one bit. Typewriter: !@#$%^&*() Apple: !"#$%&'() Digits: 1234567890 https://en.wikipedia.org/wiki/Bit-paired_keyboard https://en.wikipedia.org/wiki/Bit-paired_keyboard >A bit-paired keyboard is a keyboard where the layout of shifted keys corresponds to columns in the ASCII (1963) table, archetypally the Teletype Model 33 (1963) keyboard. This was later contrasted with a typewriter-paired keyboard, where the layout of shifted keys corresponds to electric typewriter layouts, notably the IBM Selectric (1961). The difference is most visible in the digits row (top row): compared with mechanical typewriters, bit-paired keyboards remove the _ character from 6 and shift the remaining &() from 7890 to 6789, while typewriter-paired keyboards replace 3 characters: ⇧ Shift+2 from " to @ ⇧ Shift+6 from _ to ^ and ⇧ Shift+8 from ' to . An important subtlety is that ASCII was based on mechanical typewriters, but electric typewriters became popular during the same period that ASCII was adopted, and made their own changes to layout.[1] Thus differences between bit-paired and (electric) typewriter-paired keyboards are due to the differences of both of these from earlier mechanical typewriters. >[...] Bit-paired keyboard layouts survive today only in the standard Japanese keyboard layout, which has all shifted values of digits in the bit-paired layout. >[...] For this reason, among others (such as ease of collation), the ASCII standard strove to organize the code points so that shifting could be implemented by simply toggling a bit. This is most conspicuous in uppercase and lowercase characters: uppercase characters are in columns 4 (100) and 5 (101), while the corresponding lowercase characters are in columns 6 (110) and 7 (111), requiring only toggling the 6th bit (2nd high bit) to switch case; as there are only 26 letters, the remaining 6 points in each column were occupied by symbols or, in one case, a control character (DEL, in 127). >[...] In the US, bit-paired keyboards continued to be used into the 1970s, including on electronic keyboards like the HP 2640 terminal (1975) and the first model Apple II computer (1977).
aronhegedus 2y ago
Was a really fun article to read/podcast to listen to. Favorite fact is that 127 is the DEL because for hole punching it removes all the info. I love those little nuggets of history
pixelbeat__ 2y ago
I wrote about ASCII and UTF-8 elegance at: https://www.pixelbeat.org/docs/utf8_programming.html https://www.pixelbeat.org/docs/utf8_programming.html
renox 2y ago
I still think that they made a big mistake in not having the letters immediately following the numbers, this would have made printing numbers in hexadecimal much more efficient.
eviks 2y ago
> The first 32 “characters” (and, arguably, the final one) aren’t things that you can see, but commands sent between machines to provide additional instructions Such a waste and no extensibility kills and claim to elegance of some shifted binary numbers, that'd the wrong end to focus your optimization efforts on
wduquette 2y ago
Regarding paper tape, our first home computer (this was in the mid-to-late 70's) had a paper tape reader and punch. I do not miss paper tape as a storage medium, but I have to say the little punched-out dots were fun to use as confetti at high school football games.
niobe 2y ago
I mean, this elegant design is just a necessity of efficient processing and is found in many places, it's throughout digital communications protocols for example. Look at IP addressing. In that sense all early computing is elegant since it had to be on the limited resources of the times.

285 points by thewub 2y ago

deleted 2y ago

[deleted]

Once I saw a case-insensitive switch in C using that pattern of letters: switch (my_char | 0x20) { case 'a': ... break; case 'b': ... break; }

mananaysiempre 2y ago
This can be made to work for ASCII and EBCDIC simultaneously for extra esoterica points: switch (my_char | 'A' ^ 'a') { case 'A' | 'a': /* ... */ break; /* ... */ } I don’t know if this is too fancy to have ever made it into real code, but I believe I’ve seen places in the ICU source that still say ('A' <= x <= 'I' || 'J' <= x <= 'R' || 'S <= x <= 'Z') instead of just ('A' <= x <= 'Z'), EBCDIC letters being arranged in those three contiguous ranges.
deleted 2y ago
[deleted]
Sharlin 2y ago
Yes, that’s very intentional and just masking (or setting) the bit is the intended way to do case-insensitive comparison of the letter range in ASCII (eg. stricmp in C), or to transform text to lower or upper case (tolower, toupper). But what’s more, ever wondered whence the control (Ctrl) key presses like Ctrl-H to backspace, or Ctrl-M for carriage return? Well, inspecting the ASCII chart it becomes evident: the Ctrl key simply masks bit 6 (0x40), turning a letter into its respective control character!
- lucasoshiro 2y ago
  Nice! I'm an emacs user, and when I use a readline-based REPL I use ctrl-M a lot. I thought it was inherited from the emacs keybindings, like many other shortcuts from GNU readline
  - jerf 2y ago
    Then an additional useful command: In the out-of-the-box emacs bindings, C-q is the "quoted insert" command. It will take the next character and directly insert it into the buffer. This is useful for things like tab or control characters where emacs would normally use the keystroke to do something else. I've been working in an email-related space lately so I've been doing a good amount of C-q C-m for inserting literal CRs, and C-q TAB for a few places where I want a literal tab in the source, in a buffer that interprets a normal TAB as a command to indentify the current row. I mention this because you can use the ASCII table to work out how to insert a particular control character with your keyboard literally, if you need to insert one of the handful of other characters you may be interested in every so often, like C-l for "form feed" (now used for "page feed" in some older printer-related contexts) or C-@ for NUL if you're doing something weird with binary files in a "text" buffer.
- flohofwoe 2y ago
  ...it's a bit of a shame that the same upper/lowercase trick doesn't apply to all UNICODE codepoints (at least those that have upper/lower variants). It seems to work for codepoints up to U+00FF, for instance: - Å (U+00C5) vs å (U+00E5) ...but above 0xFF lowercase follows uppercase: - Ă (U+0102) vs ă (U+0103) Typical for UNICODE though, nothing makes sense ;)
  - Findecanor 2y ago
    That's because U+00A0–U+00FF are encoding an earlier character set: "ISO Latin-1" (ISO 8859-1), itself based on DEC's "Multinational Character Set". The upper/lowercase trick does not apply to ß/ÿ but does in MCS where Ÿ/ÿ are at a different pair of code points. ISO Latin-1 was the character set on many Unix systems, Amiga OS, MS-Windows (as "Windows-1252" with extra chars), and was for many years the default character set on the web.
  - Rygian 2y ago
    In Unicode, there's no universal 1:1 mapping between cases. Lowercase "I" could very well be "ı" (lowercase dotless i) if you're typing Turkish.

dwheeler 2y ago

The encodings we use today have a surprisingly deep and complex history. For more, see: "The Evolution of Character Codes, 1874-1968" https://ia800606.us.archive.org/17/items/enf-ascii/ascii.pdf https://ia800606.us.archive.org/17/items/enf-ascii/ascii.pdf

rrwo 2y ago
Thanks for posting that. People tend to overlook that the technologies we use today have a much older history.

jiveturkey 2y ago

ebcdic is also quite elegant https://news.ycombinator.com/item?id=13543715 https://news.ycombinator.com/item?id=13543715

gerdesj 2y ago
Its shit if you don't routinely speak or write English. On those grounds, I'll decry it as not only shit but purposely shit. OK a bit over the top ... the designers of EBCSDIC had a rather tight set of constraints to deal with, none of which included: "be inclusive". Again, if I really had to be charitable (I looked after a System/36, back in the day), the hardware was rather shit too, sorry ... constrained. Yes constrained. Why should six inch fans fire up reliably after a few years of use and not need a poke after an IPL? No real dust snags and I carefully sprayed some WD40 on the one that I could get at. I have modern Dells and HPs in horrid environments that do better with shitty plastic fans. EBCDIC is not elegant at all unless excluding non English characters in an encoding system is your idea of elegant. According to this: https://en.wikipedia.org/wiki/EBCDIC https://en.wikipedia.org/wiki/EBCDIC it expended loads of effort with dealing with control eg: "SM/SW" instead of language. ASCII and EBCDIC and that basically say: fuck you foreigners! We now have hardware that is apparently capable of messianic feats. Let's do the entirety of humanity some justice and really do something elegant. It won't involve EBCDIC.
theamk 2y ago
It's really not. In base-2 machines, the letters are mixed with punctuation, which is pretty horrible design which makes simple things complex, and does not actually bring anything new to the table. In BCD machines it is slightly better, except letters aren't contiguous either - row 0 is bad, but it's the extra space between R and S which is really ugly. And it's unusable with BCD operations anyway, as high nibble values are used extensively. Naive sorting simply does not work... lowercase before uppercase, punctuation in the middle of the alphabet, numbers after letters. I see no elegance there, it's like the worst example of legacy code.
- jiveturkey 2y ago
  It was designed for a specific purpose ... elegance in context

Dwedit 2y ago

Many old NES/SNES games had a simpler character encoding system, with 0-9 and A-Z at the beginning of the table. No conversion require to display hex.

deleted 2y ago

[deleted]

ggm 2y ago

man ascii is never far from my fingers. combined with od -c and od -x it gets the job done. I don't think as fluently in Octal as I used to. Hex has become ubiquitous.

fsckboy 2y ago
you mean ? ascii
- ggm 2y ago
  No I don't -I live in a different universe to you: % (uname; cd /usr/ports; ls -d */ascii) FreeBSD zsh: no matches found: */ascii % which ascii ascii not found % It's the same on OSX and debian by default doesn't install that command. If you live inside a POSIX/IEEE 1003 system and want to know the ascii table reliably then the command I run is the one which works. If your distribution doesn't ship manuals by default you have bigger problems.
  - amszmidt 2y ago
    ”man ascii” has as much guarantee to work on a POSIX system as a command called ”ascii” seeing neither (specifically a man page called ”ascii”) are part of the standard. So you will either get command not found, or man page not found.
    - account42 2y ago
      Neither being guaranteed does not mean both have the same likelyhood of existing. The man page comes preinstalled on most modern non-embedded POSIX systems. The command does not.
- bandie91 2y ago
  man 7 ascii

transfire 2y ago

One downside of ASCII is the lack of two extra “letters” (whatever they might be, e.g. perhaps German ß), as it makes it impossible to represent base 64 alphanumerically. So we ended up with many alternatives picking two arbitrary punctuation marks.

zaxomi 2y ago
Remember that ASCII is short for "American Standard Code for Information Interchange". base64 was proposed in 1987 to work with ASCII, EBCDIC, and other character encodings.

jolmg 2y ago

> So when you’re reading 7-bit ASCII, if it starts with 00, it’s a non-printing character. Otherwise it’s a printing character. > The first printing character is space; it’s an invisible character, but it’s still one that has meaning to humans, so it’s not a control character (this sounds obvious today, but it was actually the source of some semantic argument when the ASCII standard was first being discussed). Hmm.. Interesting that space is considered a printing character while horizontal tab and newline are control characters. They're all invisible and move the cursor, but I guess it makes sense. Space is uniquely very specific in how the cursor is moved one character space, so it's like an invisible character. Newline can either imply movement straight down, or down and to the left, depending on a configuration or platform (e.g. DOS vs UNIX line endings). Horizontal tab can also move you a configurable amount rightwards, and perhaps it might've been thought a bit differently, given there's also a vertical tab, which I've got no idea on how it was used. Maybe it's the newline-equivalent for tables, e.g. "id\tcolor\v1\tred\v2\tblue\v" or something like that. Interesting also that BS is a control char while DEL is a printing(?) char. I guess that's because BS implies just movement leftwards over the text, while DEL is all ones like running a black sharpie through text. Guess that's what makes it printing. Wonder if there were DEL keys on typewriters that just stamped a black square, and on keypunchers that just punched 7 holes, so people would press "backspace" to go back then "delete" to overwrite. I've used ASCII a lot, but even after so many years, I'm getting moments where it's like "oh this piece isn't just here, it needs to be here for a deep reason". It's like a jigsaw puzzle.

pwg 2y ago
You also have to keep in mind the "interface" for 1962-1968. The printer teletype machine. The "control codes" were to "control" the printhead. So "carriage return" meant move the "print carriage" back to the left margin. "New line" meant move the paper platen one line height of rotation to move the paper to the next line. In that context, "back space" was "move print head one space left" (rather more like a "reverse space"). The article does mention that there was some debate about whether space should be considered "printable", but if you consider a mechanical printer, as the head is moving to the right and banging out characters onto the paper, the spaces between words do, sort of, look like "printables" (of a sort, a "print nothing" character as it were). Tab's being control characters then make a bit more sense, in that they cause the printhead to jump some fixed distance to the right. The article stated why DEL is where it is (all ones) -- so that for punched paper tape, one could get a punch-out of every position, which was then interpreted as "nothing here" by the tape reading machine. As for typewriters, no, none had a "black box" blot out key. Correction (for typewriters without built in correction tape) was one of: retype the page, apply an eraser (and hopefully not damage the paper surface too much) then retype character and continue, or apply correction fluid (white-out) and retype character and continue. For those typewriters with built in correction tape options (at least some IBM Selectric models, possibly more) the typewriter would retype the character using the "white-out" ribbon, then retype the replacement character using the normal "typewriting" ribbon.

BobbyTables2 2y ago

I always lament that since at least 1980s or so, it seems the vast majority of the control characters were never used for their intended purpose. Instead, we crudely use commas and tabs as delimiters instead of something like RS (#30).

thaumasiotes 2y ago
That's because the intended purpose is either useless (for machine control characters) or useless and logically impossible (for delimiters). What do you do if you have a record that includes a record separator character? Given that you have this problem anyway, why do you want a character dedicated to achieving the same thing that a comma achieves?
- penteract 2y ago
  The record separator isn't on people's keyboards, so it's less likely to show up where it's not expected. Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. Of course, the fact that record separators aren't on keyboards is probably why CSVs use commas.
  - thaumasiotes 2y ago
    > Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. No, they'll be right exactly as often, 0% of the time. But their mistake will show up less frequently, causing more problems when it does. As soon as it's possible for some of your data to come from someone else's dataset, you're guaranteed to have to accommodate record separators within your data as well as within the metadata. You're better off using a system that plans for this inevitability than one that pretends it can't happen at all.

th0ma5 2y ago

I heard someone describe the ASCII table as a state machine. Guess I could understand that as a state machine needed to parse it? This is surprisingly hard to search for but I was wondering if anyone knows what they were talking about.

EvanAnderson 2y ago
They might be talking about using escape sequences to map additional codepoints into ASCII. It was designed to be extensible. See: https://web.archive.org/web/20150810075144/http://bobbemer.com/ESCAPE.HTM https://web.archive.org/web/20150810075144/http://bobbemer.c...
kevin_thibedeau 2y ago
Bespoke hardware for text handling isn't a thing these days but would have been in the 60's and 70's. A table layout that can be easily decoded in hardware simplifies the necessary circuitry for responding to control characters or converting binary numbers to/from decimal when the microprocessor hadn't been invented yet.
gumby 2y ago
It was originally implemented in actual hardware (rods and bars). Just look inside a teletype, like a KSR-23 (pre ascii, but similar)

deleted 2y ago

[deleted]

EvanAnderson 2y ago

I would be remiss not to post a link to the late Bob Bemer's[0] website. https://web.archive.org/web/20150801005415/http://bobbemer.com/ https://web.archive.org/web/20150801005415/http://bobbemer.c... He was considered the "father of ASCII". Hr wrote very well and gives clear explanations for the motivations behind the design of ASCII. [0] https://en.m.wikipedia.org/wiki/Bob_Bemer https://en.m.wikipedia.org/wiki/Bob_Bemer

thristian 2y ago

> That, I’m afraid, is because ASCII was based not on modern computer keyboards but on the shifted positions of a Remington No. 2 mechanical typewriter – whose shifted layout was the closest compromise we could find as a standard at the time, I imagine. According to Wikipedia¹, American typewriters were pretty consistent with keyboard layout until the IBM Selectric electric typewriter. Apparently "small" characters (like apostrophe, double-quote, underscore, and hyphen) should be typed with less pressure to avoid damaging the platen, and IBM decided the Selectric could be simpler if those symbols were grouped on dedicated keys instead of sharing keys with "high pressure" symbols, so they shuffled the symbols around a bit, resulting in a layout that would look very familiar to a modern PC user. Because IBM electric typewriters were so widely used (at least in English speaking countries), any computer company that wanted to sell to businesses wanted a Selectric-style layout, including the IBM PC. Meanwhile, in other countries where typewriters in general weren't so popular or useful, the earliest computers had ASCII-style punctuation layout for simplicity, and later computers didn't have any pressing need to change, so they stuck with it. Japanese keyboards, for example, are still ASCII-style to this day. ¹: https://en.wikipedia.org/wiki/IBM_Selectric#Keyboard_layout https://en.wikipedia.org/wiki/IBM_Selectric#Keyboard_layout

sixothree 2y ago
I never realized my first computer used ascii directly for the shifted number keys. https://en.wikipedia.org/wiki/TRS-80_Color_Computer#/media/File:TRS-80_Color_Computer_1.jpg https://en.wikipedia.org/wiki/TRS-80_Color_Computer#/media/F...
- Mountain_Skies 2y ago
  So many things on the CoCo turned out to be the way they were for cost saving reasons. Tandy was good at saving pennies everywhere it could. When I took 'Typing' in high school, my muscle memory was in a constant fight between the IBM Selectric layout of the typewriters at school and the CoCo at home.
- kps 2y ago
  https://en.wikipedia.org/wiki/Bit-paired_keyboard https://en.wikipedia.org/wiki/Bit-paired_keyboard

kragen 2y ago

unfortunately this page is based on mackenzie's book. mackenzie is the ibm guy who spent decades trying to kill ascii, promoting its brain-damaged ebcdic as a superior replacement (because it was more compatible, at least if you were already an ibm customer). he spends most of his fucking book trumpeting the virtues of ebcdic actually bob bemer more or less invented ascii. he was also an ibm guy before mackenzie's crowd pushed him out of ibm for promoting it. he wrote a much better book about the history of ascii which is also freely available online, really more a pamphlet than a book, called "a story of ascii": https://archive.org/details/ascii-bemer/page/n1/mode/2up https://archive.org/details/ascii-bemer/page/n1/mode/2up tom jennings, who invented fido, also wrote a history of ascii, called 'an annotated history of some character codes or ascii: american standard code for information infiltration'; it's no longer online at his own site, but for the time being the archive has preserved it: https://web.archive.org/web/20100414012008/http://wps.com/projects/codes/index.html https://web.archive.org/web/20100414012008/http://wps.com/pr... jennings's history is animated by a palpable rage at mackenzie's self-serving account of the history of ascii, partly because bemer hadn't really told his own story publicly. so jennings goes so far as to write punchcard codes (and mackenzie) out of ascii's history entirely, deriving it purely from teletypewriter codes—from which it does undeniably draw many features, but after all, bemer was a punchcard guy, and ascii's many excellent virtues for collation show it as dwheeler points out, the accomplished informatics archivist eric fischer has also written an excellent history of the evolution of ascii. though, unlike bemer, fischer wasn't actually at the standardization meetings that created ascii, he is more careful and digs deeper than either bemer or jennings, so it might be better to read him first: https://archive.org/details/enf-ascii/ https://archive.org/details/enf-ascii/ it would be a mistake to credit ascii entirely to bemer; aside from the relatively minor changes in 01967 (including making lowercase official), the draft was extensively revised by the standards committees in the years leading up to 01963, including dramatic improvements in the control-character set for the historical relationship between ascii character codes and keyboard layouts, see https://en.wikipedia.org/wiki/Bit-paired_keyboard https://en.wikipedia.org/wiki/Bit-paired_keyboard

gumby 2y ago

I wish the author had included the full ascii chart in 4 bits across / 4 bits down. You can mask a single bit to change case and that is super obvious that way. The charts that simply show you the assignments in hex and octal obscure the elegance of the design.

AdamH12113 2y ago
The third and fourth columns of the table are only a single bit apart from each other. If you mentally swap the first two columns, you get a Gray code ordering of the most significant bits, which is pretty close to what you're looking for.
- gumby 2y ago
  Found it: https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code_chart.png https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code...
kalleboo 2y ago
It was at some point looking at a chart like that where it also dawned on me where the control codes like ^D, ^H, ^[ etc came from
- kr2 2y ago
  I was going to ask you to please explain as I didn't understand, but I am guessing you are talking about the same thing as this comment[1] right? That's super cool https://news.ycombinator.com/item?id=41042570 https://news.ycombinator.com/item?id=41042570
  - gumby 2y ago
    Yes, though ^m, ^[ et al aren’t so much elegant as coincidental; but look at A and a, for example. I found the chart I was looking for: https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code_chart.png https://en.wikipedia.org/wiki/ASCII#/media/File:USASCII_code...

netcraft 2y ago

I've searched off and on for a great stylistic representation of the ASCII table, id love a poster to hang on my wall, or possibly even something I could get as a tattoo.

yawl 2y ago

I also wrote a chat novel about ASCII: https://www.lostlanguageofthemachines.com/chapter2/chat https://www.lostlanguageofthemachines.com/chapter2/chat

userbinator 2y ago

You might be familiar with carriage return (0D) and line feed (10) You mean 0D and 0A, or 13 and 10, but that mix of base really stood out to me in an otherwise good article. I'm one of numerous others who have memorised most of the base ASCII table, and quite a few of the symbols as well as extended ASCII (CP437), mainly because it comes in handy for reading programs without needing a disassembler. Those who do a lot of web development may find the sequence 3A 2F 2F familiar too, as well as 3Ds and 3Fs. I can see the rationale for <=> being in that order, but [\] and {|} are less obvious, as well as why their position is 1 column to the left of <=>.

yardshop 2y ago
> You mean 0D and 0A, or 13 and 10 He fixed it.

KingOfCoders 2y ago

For everyone who doesn't need ä,ü,ö. Or software that needs to take ä,ü,ö. For everyone else, UTF is a blessing.

bigstrat2003 2y ago
Which, given the people who designed this and the time they were designing for, was most of them (and most of their audience). Don't confuse "this old standard doesn't adequately cover all cases today" with "this old standard sucked at the time".
- eviks 2y ago
  Don't confuse "this old standard sucked at the time for missing the obvious at the time" with "I can come up with some excuse"
lmm 2y ago
> For everyone else, UTF is a blessing. Except people who want to use Japanese and not have it render weirdly, something that was easy in any internationalised software that used the traditional codepage system, but is practically impossible in Unicode-based software.
- Retr0id 2y ago
  Where can I learn more about this issue?
  - zokier 2y ago
    https://en.wikipedia.org/wiki/Han_unification https://en.wikipedia.org/wiki/Han_unification
  - p_l 2y ago
    Probably referring to so-called "Han unification" which tried to use same codepoints for different glyphs to reduce code space for ideograms derived from Chinese ones. But that only causes confusion because you need to provide external information which way to interpret them, just like a code page

blahedo 2y ago

Another piece of elegance: by putting the uppercase letters in the block beginning at 0x40 (well, 0x41) it means that all the control codes at the start of the table line up with a letter (or one of a small set of other punctuation: @[\]^_), giving both a natural shorthand visual representation and a way to enter them with an early keyboard, by joining the pressing of the letter with... the Control key. Control-M (often written ^M) is carriage return because carriage return is 0x0D and M is 0x4D.

WalterBright 2y ago

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come on!) 8. you vote for my made-up emoticon, and I'll vote for yours!

Retr0id 2y ago
Language itself is a pile of ugly graffiti and ramshackle addons. It would be weird if Unicode didn't reflect this.
jart 2y ago
Hey at least we got the astral planes. https://justine.lol/dox/unicode.txt https://justine.lol/dox/unicode.txt
p_l 2y ago
How to say you don't know what Unicode is for without saying it. 1, 2, 4, 5, 6, and, unfortunately, 8, all fall under "ability to encode written text from all human languages". And that includes historical. Some of the issues (5 & 6) are due semantic difference even if the resulting glyph looks the same. Unfortunately you can't expect programmers to understand pesky little thing like languages having different writing, so you end up with normalisation to handle the fact that one system sent "a + ogonek accent" and another (properly) sent "a with ogonek" (these print the same but are semantically different!), and now you need to figure out normalisation in order to be able to compare strings. 7. just like 8 are down to proposal of specific new forms of writing to add to Unicode. Elvish had one since 1997 but only now got a tentative "we will talk about it". Klingon, which is IIRC more complete language including native speakers (...weird things happened sometimes) does not have outside of private use area. Emojis were added because they were used with incompatible encodings first, even before unicode happened, and without including something like SIXEL into unicode they were unrepresentable (and with SIXEL would lose semantic information)

zokier 2y ago

I think that adopting ASCII as the general purpose text encoding was one of the great mistakes of early computing. It originated as control interface for teletypes and such, and that's arguably where it should have remained. For storing and processing (plain) text ASCII doesn't really fit that well, control characters are a hindrance and the code space would have been useful for additional characters. The ASCII set of printables was definitely a compromise formed by the limited code space.

kstenerud 2y ago
It's one of the greatest triumphs of early computing. Not only did it harmonize text representation and transmission in a backwards compatible manner; the fact that they deliberately kept it 7 bit for so long also helped for developing a sane set of other language character sets (ISO-8859), and paved the way for a smooth transition to Unicode (UTF-8) - which is now the dominant encoding worldwide.
- ddingus 2y ago
  Yes, seconded easily
hackit2 2y ago
Yeah you were not around when a kb of memory took up half your room. Looking back it doesn't make sense but at the time a byte was what-ever you wanted it to be. Considering number of characters in English language is 26, it is reasonable for a byte to be 5 bits, giving you a total of 32 possible states. Which leaves you with 6 values which could be used as control characters. how-ever lets not forget there are 7,164 other languages of the world, and they all have their own unique way of doing things. Oh yeah, lets not forget that at the time you had other nationalistic countries/territories/people with their own superior technology all vying for the top position, all well trying to out do each other. Then you also had manipulative monopolies/trade embargo's and wars. It isn't perfect but people aren't perfect.
ddingus 2y ago
No way! No amount of extra characters was going to address what Unicode did. ASCII was not a mistake at all. Adopting it unified what was surely going to be a real mess. At the time it made sense, and the control functions were needed. Still are.

augusto-moura 2y ago

Useful tip, on linux (not sure about other *nixes) you can view the ascii table by opening its manpage: man ascii It's been useful to me more than once every year, mostly to know about shell escape codes and when doing weird character ranges in regex and C. It can be a bit confusing, but the gist is that you have 2 chars being show in each line, I would prefer a view where you see the same char with shift and/or ctrl flags, but you can only ask so much

dailykoder 2y ago
Damn, thanks! Why the hell did I never try this? Maybe because typing ascii table into my favorite search engine and clicking one of the first links was fast enough
- omnicognate 2y ago
  I used to do that until the experience became degraded enough, reflecting the general state of the web, that I took the time to look for a better way and found `man ascii`.
bell-cot 2y ago
Similar in FreeBSD. It has octal, hex, decimal, and binary ASCII tables, along with the full names of the control characters.
INTPenis 2y ago
The reason I know this is because in 2004 I was squatting in an apartment with no TV and no internet. So each day after work I would go home and just read manpages for fun. Ended up learning ipfw through the firewall manpage on FreeBSD, and using my skills to setup and manage an IPFW at work. It's amazing how much you get done with no TV and no internet. Also played a lot of nethack.
- w0m 2y ago
  I learned vim proper by reading :help on an eeepc while flying back and forth over the Atlantic alone one year.

snvzz 2y ago

The ASCII table is defective; it is missing a dedicated code for newline. CR and LF aren't dedicated, and have precise cursor movement meanings, rather than being a logical line ender. There was a proposal in the 80s to reassigning the -otherwise useless- VT (vertical tab) character for the purpose. Unfortunately unfruitful.

bregma 2y ago
A separate control character was not needed to indicate where your Hollerith string ended: it ended at the end of the Hollerith string. If you wanted to render a Hollerith string onto print media, you'd often want to feed the line and then return the carriage before printing the next Hollerith string. Of course, that wasn't strictly necessary if you were using a line printer, which would just print the line and advance. The filesystems I used had 5 kinds of file: random-access, sequential, ISAM, Fortran-carriage-control and carriage-return-carriage-control. The only people who used the latter were the eggheads that used that new-fangled C programming language brought over from Bell Lab's experimental Unix system. You're probably just looking for the record separator (036). If you are storing multiple text records and a block of memory, that would be the ideal ASCII code to separate them.
- snvzz 2y ago
  >You're probably just looking for the record separator (036). Certainly not for records that are made of multiple lines.
Gormo 2y ago
> Unfortunately unfruitful. Fortunately unfruitful, since if it had gained adoption, there'd be a mix of three different line endings (and combinations thereof) in widespread use, instead of two.
- snvzz 2y ago
  >widespread use We are talking about the early 80s at worst. It would have consolidated by now.

1vuio0pswjnm7 2y ago

Mentioned in footnote 7: https://ia601808.us.archive.org/2/items/mackenzie-coded-char-sets/Mackenzie_CodedCharSets.pdf https://ia601808.us.archive.org/2/items/mackenzie-coded-char...

red_admiral 2y ago

The "16 rows x 8 columns" version, with the lowercase letters added, seems the most elegant one to me because it makes the internal structure of the thing visible. For example, to lowercase a letter, you set bit 6; a decimal digit is the prefix 011 followed by the binary encoding of the digit etc. It also makes clear why ESC can be entered as `^[` or ENTER (technically CR) as `^M` on some terminals (still works in my xterm), because the effect of the control key is to unset bits 6 and 7 in the original set-up. Of course you can color in the fields too, if you want.

johanneskanybal 2y ago

Kind of hard to read something where the author considers every non-english languages equally worthy to emoji’s.. It was good in the 50’s but was important like 4-5 decades too long.

georgehotelling 2y ago

Dark grey #303030 text on slightly darker grey #1B1C21 background is really hard to read. Maybe I'm just getting old, but I also assume the audience for a blog post about the ASCII table was born in a year that starts with 19.

Retr0id 2y ago
The background is white on my machine, are you using some kind of extension to force "dark mode"?
- georgehotelling 2y ago
  I'm using pi-hole, uBlock Origin, and Privacy Badger on Firefox. I checked my network tab before complaining and didn't see any resources that failed to load.

bloak 2y ago

Vaguely related: Apart from £ and €, a typical GB keyboard has a couple of non-ASCII characters printed on it: ¬ and ¦. The key labelled ¦ is usually mapped to |, but the key labelled ¬ often gives you an actual ¬, though I can't remember many occasions on which I've wanted one of them. Apparently the characters ¬ and ¦ are in EBCDIC.

PaulHoule 2y ago

Beats EBCDIC https://en.wikipedia.org/wiki/EBCDIC https://en.wikipedia.org/wiki/EBCDIC On the 4th floor of my building the computer systems lab has a glass front that has what looks like a punch card etched in frosted glass but if you look closer it was made by sticking stickers on the glass. I made a "punchcard decoder" on a 4x6 card to help people decode the message on the wall https://mastodon.social/@UP8/112836035703067309 https://mastodon.social/@UP8/112836035703067309 The EBCDIC code was designed to be compatible with this encoding which has all sorts of weird features, for instance the "/" right between "R" and "Z"; letters don't form a consecutive block so testing to see if a char is a letter is more complex than in ASCII. I am thinking of redoing that card to put the alphabet in order. A column in a punched card has between 0 to 3 punches, 0 is a space, 1 is a letter or a symbol in the first column, if one of the rows at the top is punched you combine that with the number of the other punched row on the left 3x9 grid. If three holes are punched one of them is an 8 (unless you've got one of the extended charsets) and you have one of the symbols in the right 3x6. Note the ¬ and ¢ which are not in ASCII but are in latin-1.

senkora 2y ago

Fun fact: sorting ASCII numerically puts all the uppercase letters first, followed by all the lowercase letters (ABC... abc...). A more typical dictionary ordering would be more like AaBbCc... (or to even consider A and a at the same sort level and only use them to break ties if the words are otherwise identical). The order used by ASCII is sometimes called "ASCIIbetical", which I think is wonderful. https://en.wiktionary.org/wiki/ASCIIbetical https://en.wiktionary.org/wiki/ASCIIbetical

NoMoreNicksLeft 2y ago
I thought the point of that was that a single bitflip makes an uppercase lower, or vice versa...
- Dylan16807 2y ago
  It can't be "the" point, because AaBbCc would also let you use a single bit to control case, the bottom bit.
  - NoMoreNicksLeft 2y ago
    The first bit was already taken at that point. The original 6bit ASCII (or whatever that predecessor was called) only did uppercase. A 7th (and later, 8th) bit was added to give it lowercase.

DonHopkins 2y ago

The Apple ][ and TTYs and other old computers had "bit pairing keyboards", where the punctuation marks above the digits were aligned with the ASCII values of the corresponding digits, different by one bit. Typewriter: !@#$%^&*() Apple: !"#$%&'() Digits: 1234567890 https://en.wikipedia.org/wiki/Bit-paired_keyboard https://en.wikipedia.org/wiki/Bit-paired_keyboard >A bit-paired keyboard is a keyboard where the layout of shifted keys corresponds to columns in the ASCII (1963) table, archetypally the Teletype Model 33 (1963) keyboard. This was later contrasted with a typewriter-paired keyboard, where the layout of shifted keys corresponds to electric typewriter layouts, notably the IBM Selectric (1961). The difference is most visible in the digits row (top row): compared with mechanical typewriters, bit-paired keyboards remove the _ character from 6 and shift the remaining &() from 7890 to 6789, while typewriter-paired keyboards replace 3 characters: ⇧ Shift+2 from " to @ ⇧ Shift+6 from _ to ^ and ⇧ Shift+8 from ' to . An important subtlety is that ASCII was based on mechanical typewriters, but electric typewriters became popular during the same period that ASCII was adopted, and made their own changes to layout.[1] Thus differences between bit-paired and (electric) typewriter-paired keyboards are due to the differences of both of these from earlier mechanical typewriters. >[...] Bit-paired keyboard layouts survive today only in the standard Japanese keyboard layout, which has all shifted values of digits in the bit-paired layout. >[...] For this reason, among others (such as ease of collation), the ASCII standard strove to organize the code points so that shifting could be implemented by simply toggling a bit. This is most conspicuous in uppercase and lowercase characters: uppercase characters are in columns 4 (100) and 5 (101), while the corresponding lowercase characters are in columns 6 (110) and 7 (111), requiring only toggling the 6th bit (2nd high bit) to switch case; as there are only 26 letters, the remaining 6 points in each column were occupied by symbols or, in one case, a control character (DEL, in 127). >[...] In the US, bit-paired keyboards continued to be used into the 1970s, including on electronic keyboards like the HP 2640 terminal (1975) and the first model Apple II computer (1977).

aronhegedus 2y ago

Was a really fun article to read/podcast to listen to. Favorite fact is that 127 is the DEL because for hole punching it removes all the info. I love those little nuggets of history

pixelbeat__ 2y ago

I wrote about ASCII and UTF-8 elegance at: https://www.pixelbeat.org/docs/utf8_programming.html https://www.pixelbeat.org/docs/utf8_programming.html

renox 2y ago

I still think that they made a big mistake in not having the letters immediately following the numbers, this would have made printing numbers in hexadecimal much more efficient.

eviks 2y ago

> The first 32 “characters” (and, arguably, the final one) aren’t things that you can see, but commands sent between machines to provide additional instructions Such a waste and no extensibility kills and claim to elegance of some shifted binary numbers, that'd the wrong end to focus your optimization efforts on

wduquette 2y ago

Regarding paper tape, our first home computer (this was in the mid-to-late 70's) had a paper tape reader and punch. I do not miss paper tape as a storage medium, but I have to say the little punched-out dots were fun to use as confetti at high school football games.

niobe 2y ago

I mean, this elegant design is just a necessity of efficient processing and is found in many places, it's throughout digital communications protocols for example. Look at IP addressing. In that sense all early computing is elegant since it had to be on the limited resources of the times.