4 ms·
Token tok; start_token(&tok); for (;;) { int c = look_next_char(); if (('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z')) {
by jstimpfle 8y ago
Token tok;
start_token(&tok);
for (;;) {
int c = look_next_char();
if (('A' <= c && c <= 'Z') ||
('a' <= c && c <= 'z')) { /* or whatever test */
consume_char();
add_to_token(tok, c);
} else {
break;
}
}
end_token(tok);
Done. There's no point in going through a weird API.
- tptacek 8y agostrcspn is ANSI C90.
- torstenvl 8y agoYou've not only hard-coded your tokenization rules inside your logic, but you've managed to make it break on anything non-ASCII. I suppose you consider isalpha() to have a weird API?
- jstimpfle 8y agoI look at it the other way: I've hard coded the reading and writing routines inside the tokenization logic. Being able to do that is exactly the point why it's so much simpler to avoid a silly API such as strcspn (or, god forbid, strtok). > non-ASCII yeah i know... Do you prefer strcspn("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVXYZ")? Do you think it's faster? If you're pedantic, you could lex (0x41 <= c && c <= 0x5A). That way at least you consistently read ASCII, even on non-ASCII implementations. But I don't care and it's less readable. > I suppose you consider isalpha() to have a weird API? Yes. I do not even understand what it does. >> isalpha() checks for an alphabetic character; in the standard "C" locale, it is equivalent to (isupper(c) || islower(c)). In some locales, there may be additional characters for which isalpha() is true-letters which are neither upper case nor lower case. Well in any case I'm sure that's not what I wanted... By the way locale is super hard to use as well. Locale is a process global property. I'm not aware of any way to pass explicit locales to library functions.
- avar 8y ago> If you're pedantic, you could lex (0x41 <= c && c <= 0x5A) 'A' v.s. 0x41 makes no difference for portability. The thing that's unportable about that is that it assumes that the characters A..Z are continuous in your character encoding, which isn't portable C. Although admittedly having to deal with EBCDIC these days is rare in anything except highly portable programs like C compilers or popular script interpreters. This is why ctype.h functions exist. Just use them.
- loeg 8y ago> 'A' v.s. 0x41 makes no difference for portability. The thing that's unportable about that is that it assumes that the characters A..Z are continuous in your character encoding, which isn't portable C. Wait, what? If C does not require A..Z to be contiguous, the distinction between 'A' and 0x41 is extremely significant to portable programs intending to parse ASCII when the native compiler encoding is whatever franken-coding doesn't have contiguous latin characters.
- zozbot123 8y agoEBCDIC famously does not have A..Z as contiguous characters, and I wouldn't describe it as a 'franken-coding' just yet - it still finds plenty of use in some places.
- avar 8y agoYes. If the problem was trying to parse ASCII consistently that would be the right solution. My response was to OPs moving the goal post to "portably parsing ASCII" in response to his suggested replacement for a C library function not being portable on non-ASCII systems, which make no sense.