8 ms·
The fact that Unicode codepoints were being passed to isspace instead of iswspace indicates the relevant code was already fubar'd. > For example, isspace(0x01f
by wahern 3y ago
The fact that Unicode codepoints were being passed to isspace instead of iswspace indicates the relevant code was already fubar'd.
> For example, isspace(0x01fe) is true. I can't figure out why this might be considered a whitespace character
Because the only valid values (independent of locale) that can be passed to isspace are 0 to UCHAR_MAX and -1/EOF, where UCHAR_MAX refers to unsigned char (usually 255), not Unicode character. Most implementations I've seen (glibc, musl, OpenBSD) index the passed value into a locale-specific array of length UCHAR_MAX + 1, possibly masking the index and/or return values. But TIL macOS (and possibly FreeBSD and NetBSD at some point, if not currently) had vestigial support for passing higher values as part of a presumably long-abandoned approach to I18N.
EDIT: FWIW, based on the glibc code (ctype/isctype.c),
int
__isctype (int ch, int mask)
{
return (((uint16_t *) _NL_CURRENT (LC_CTYPE, _NL_CTYPE_CLASS) + 128)
[(int) (ch)] & mask);
}
where isspace(c) seems to be translated to __isctype(c, _ISspace) there's a good chance the array is being overflowed. Without looking further (glibc isn't the easiest code to grok), I'd guess the array size is probably 128 + UCHAR_MAX with the offset of 128 (instead of 1) to handle the common case, especially on systems where char is signed, of people passing in negative values, though that only works for a locale like ASCII where -1/EOF and 255/(unsigned char)-1 aren't ambiguous.
- dvh 3y agoSeems like instead of using boolean result better API should return tri-state: space, no space, bad request
- bitwize 3y agoSo... True, False, and File Not Found?
- teddyh 3y agoReference: https://thedailywtf.com/articles/What_Is_Truth_0x3f_ https://thedailywtf.com/articles/What_Is_Truth_0x3f_
- rm445 3y agoReminds me of a joke in a John Meaney novel, a physicist discovers faster-than-light travel through a dimension with unusual properties, gaining inspiration from Java's booleans having three states (true, false and NullPointerException).
- Phrodo_00 3y agoHate to point it out, but java booleans can only be true or false. Booleans (capital B) can be true, false or null.
- dfox 3y agoMethod that is declared as only ever returning bool can also raise any exception that is an instance of java.lang.RuntimeException (which includes NullPointerException) this somewhat bizzare design which combines unchecked and checked exceptions is a source of major discussions since at least late 90's and has much to do with the current concept of null-safety.
- hedora 3y agoYeah, but using a raw boolean or Boolean is not considered good practice. They’re up to at least four states now: null, None, Some(TRUE) and Some(FALSE) Of course, if you want more than that, you can have unboundedly many by invoking new Boolean(), which guarantees its return value is unique according to ==
- TeMPOraL 3y agoThis is already the case; "bad request" is usually returned as SIGSEGV / 0xC0000005 and similar.
- asveikau 3y agoNo. Bad request here would be a recoverable, handleable error that doesn't tank the process. You are describing a failure mode for a bad pointer dereference, which generally should not be handled in-process and should take down the program. It is not reasonable for isspace(), a simple function call with no pointer arguments, to fault like that, absent unlikely events such as a hardware issue or a bad stack pointer.
- TeMPOraL 3y ago> It is not reasonable for isspace(), a simple function call with no pointer arguments, to fault like that Well, if it internally uses a lookup table and doesn't do bounds checking, then it can fail this way, and that failure will propagate to the caller. > You are describing a failure mode for a bad pointer dereference, which generally should not be handled in-process and should take down the program. Maybe this is where we went wrong? Bad pointer access isn't fundamentally different from doing a bounds check and throwing an exception. Either of them will tank the process if not handled up the stack. But when you do a bounds check solely to prevent a bad pointer access crash, you're effectively re-doing the work the OS already does for you, paying a cost at runtime just to switch to a slightly different semantics. Maybe instead of doing that, we should have the platform provide better granularity of protection (as to e.g. avoid bad pointer access targeted to overwrite some other memory), and lean on it, instead of bolting extra layers of the same thing, just Invented Here on top?
- deredede 3y agoYou can't exactly lean on the OS to detect bad pointers references because it will only work some of the time. The OS doesn't know where your array end, so the error would have to depend on whether the access is outside the memory allocated to the process. A function specification of "if the input is outside valid range, either return garbage or raise an exception" provides no value over "either return garbage or crash the program", because there is still the possibility of returning garbage.
- JdeBP 3y agoFreeBSD layers narrow and wide character typing on top of a single common mechanism based upon 32-bit signed "runes". Basically, the 256 narrow characters are treated as the first 256 characters in the Unicode BMP, and an accident of implementation allows one to pass in other Unicode code points to the narrow character functions, given that the int and wint_t types are designed to be trivially convertible to a "rune".