6 ms·
Discovering one bug after another in the UTF-8 decoding logic in OpenBSD
- yakubin 4y agoIt's weird to me that text encoding is put in the domain of the kernel. But I guess it's the way it needs to be until the world moves to microkernels.
- layer8 4y agoThis seems to be about the system console built into the kernel.
- yakubin 4y agoYeah, I got that part. I've just assumed that would be handled by a server process in a microkernel architecture. But I've just checked seL4 and it has in-kernel support for the console as well. That's not what I expected.
- MisterTea 4y agoThe system terminal has to be in-kernel otherwise you would not see kernel prints during boot, before any fs (and servers launched from it) is mounted.
- hinkley 4y agoEven with a microkernel, there's a lot of power in having a text console even when half the system is not functioning. If this code exists in a separate process, a lot of the handling bugs still need to be fixed to make the OS usable.
- snvzz 4y agoSerial port is trivial.
- Lt_Riza_Hawkeye 4y agookay... and something needs to be able to decode the text sent to it over the serial port. All the bugs pointed out in the article also happen if you use the serial port console
- comex 4y agoYes, and that “something” can be an app like gnome-terminal running on a full-fat text rendering library and GUI framework… on a completely different computer. That software could hypothetically have the same bug, but probably not, and at any rate it‘s not taking up space in the core of a microkernel.
- matja 4y agoThe 120k+ lines of code in drivers/tty/serial in Linux suggests that maybe it's not _that_ trivial.
- jamal-kumar 4y agoYeah, what's even weirder is when it's so integral to the kernel that it can lead to crashes. I've seen more than one case now where someone just fuzzing around on text engines ends up taking down an entire system, at least a couple on mac osx and ios alone. Dangerous stuff. [1] [2] Linux might not be the most performant in the text engine game but at least it runs in userland. [1] https://arstechnica.com/gadgets/2013/08/rendering-bug-crashes-os-x-and-ios-apps-with-string-of-arabic-characters/ https://arstechnica.com/gadgets/2013/08/rendering-bug-crashe... [2] https://www.engadget.com/unusual-buggy-character-string-apple-devices-crash-120521234.html https://www.engadget.com/unusual-buggy-character-string-appl...
- phoehne 4y agoI've been waiting for a "true" microkernel OS since 1980s.
- lproven 4y agoQNX and Minix 3 don't count?
- hinkley 4y agoMy eyes are bleeding. Reader mode to the rescue. Always easier to see someone else's problems than our own, I suppose.
- mouse_ 4y agoI think they're going for something like https://old.reddit.com/r/mildlyinfuriating/ https://old.reddit.com/r/mildlyinfuriating/
- volkadav 4y agoidk, I think the site's design is kind of endearing even if it isn't super ergonomic. Maybe I'm just old and nostalgic for the wild west creativity of the early web before everyone slapped bootstrap on wordpress or whatever and started looking the same. :)
- jameshart 4y ago'endearing' is about right. It's programmer art - the kind of stylesheet someone puts together to demonstrate all the features of their CSS engine, without concern as to its suitability to any particular visual purpose.
- notfed 4y agoIn case anyone thinks this is just snark, it's not. The font shadow legitimately hurts my eyes and I am unable to read this article.
- spc476 4y agoAnd aside from one <svg> block and one <table> block, all the tags are either <div>s or <spans>s. That's ... impressive.
- hinkley 4y agoFont shadow misuse and multiple saturated colors are the most common ways to make something painful to look at. Tiny fonts too. This one used shadows and flirts with too much saturated color.
- smm11 4y agoI was just thinking the old internet was better, then I visited the link. Not that I'm not interested, but ...
- lsllc 4y agoIt loads too quickly! Is there a "baud-rate" CSS selector that could be set to 19.2K? (unless you were lucky enough to have one of those fancy 56K modems!).
- dhosek 4y agoI think I was on 56K pretty early. Sending full-page 2400dpi bitmaps as PostScript files to the service bureau to generate film was a lengthy process.
- muyuu 4y agoit's not old enough it uses fancy text effects you wouldn't have used in the 90s, and assets way too heavy for the 90s adjust those two things to the 90s and the site is pretty much perfect for what it does
- jamal-kumar 4y agoI love this site, aside from the design the articles on it are good reads. I guess OpenBSD people have a hilarious idea of design sometimes. I remember when de raadt and some others were making slides for conferences in MS comic sans just to "annoy design nerds" (not my wording IIRC) for a while
- messe 4y ago> I remember when de raadt and some others were making slides for conferences in MS comic sans just to "annoy design nerds" (not my wording IIRC) for a while I'm pretty sure that's still a thing, and as somebody who nerds out on typography (my undergrad thesis used the Tufte LaTeX package), I fucking love it.
- vfclists 4y ago> aside from the design the articles on it are good reads The design is perfectly fine.
- steponlego 4y agoIt’s horrid.
- somat 4y agoTangentially, I always found it strange that five and six byte utf-8 sequences are illegal. My naive thought is that it provided a good amount of future proofing for when Unicode needs to expand again.
- dhosek 4y agoUnicode was originally conceived of as a 16-bit encoding. Han unification was necessary to squeeze everything into 16 bits, but it (kind of) worked. Going from 16 bits to 20 bits¹ might not seem like a big increase, but remember that this is an exponential progression, and not a linear one. Those four bits increase the capacity of Unicode by a factor of 16. The amount of empty space is much greater than we’re likely to ever use. Furthermore, there are costs (albeit not necessarily large ones) to having those empty code points—even with efficient use of space using 2-level lookup tables, increasing the capacity of Unicode beyond its current size would add time and space complexity to code for doing things like identifying character codes, grapheme boundaries, case folding, sorting, etc. ⸻ 1. Technically, its 21 bits, but since the maximum value for a code point is 0x10ffff, we can round down to 0xfffff—which means we’re treating the original 16-bit capacity of Unicode as a rounding error!
- toast0 4y agofive and six byte sequences are defined as illegal, because they encode code points that don't fit in utf-16. Of the 1,112,064 assignable code points allowed by UTF-16, we've only assigned 149,186 in the past 30 years, so we've probably got a couple centuries before we'll run out. Deal with it then.
- bentley 4y agoI’m surprised that you were downvoted, because you’re totally right. UTF‐16 can represent almost a 21‐bit range—yes, greater than 16—thanks to surrogate pairs. The largest codepoint representable in UTF‐16 is U+10FFFF. That’s why UTF‐8 dropped the 5‐ and 6‐byte sequences, why even UTF‐32 is limited to 21 bits, and why U+D800 to U+DFFF (the range UTF‐16 uses to encode surrogate pairs) are explicitly disallowed from being represented in UTF‐8 and UTF‐32.
- camgunz 4y agoI love the site design, and the other themes are cool too!