9 ms·
Harmful Consequences of Postel's Maxim
- MichaelBurge 9y agoI've worked a few APIs that are "liberal in what they emit, and conservative in what they accept", which gets you the worst of both worlds.
- dbcurtis 9y agoThat seems to be a design principle at Apple.
- robertlagrant 9y agoif(vendor.name == "Apple") open(arms);
- nabla9 9y agoI like /r/netsettelr's comments in reddit. https://www.reddit.com/r/programming/comments/6u1jq2/the_harmful_consequences_of_postels_maxim_an/dlpe1hx/ https://www.reddit.com/r/programming/comments/6u1jq2/the_har...
- kutkloon7 9y agoFrom the page which the top comment from two years [1] ago linked to: "This statement is based upon a terrible misunderstand of Postel's robustness principle. I knew Jon Postel. He was quite unhappy with how his robustness principle was abused to cover up non-compliant behavior, and to criticize compliant software. Jon's principle could perhaps be more accurately stated as 'In general, only a subset of a protocol is actually used in real life. So, you should be conservative and only generate that subset. However, you should also be liberal and accept everything that the protocol permits, even if it appears that nobody will ever use it.'" [1] https://news.ycombinator.com/item?id=9827669 https://news.ycombinator.com/item?id=9827669
- whipoodle 9y agoInteresting! That is definitely different from how the maxim is commonly understood. Edit: actually, some of the replies in the thread you linked disagree with this remembrance, and state that it really did mean what we take it to mean today.
- jcranmer 9y agoThe citations for the replies don't really back it up. The context for the various iterations of Postel's Law generally suggest that they're referring to the possibility that you might be seeing the result of mismatching versions of a specification. The later iterations also give reference to an explicit example of what they mean: don't assume that enumerations in the specification are closed (i.e., assume that future revisions may add additional enumerations). There is absolutely no evidence that he is advocating trying to parse slop at all.
- Animats 9y agoHTML suffers badly from this. Even basic lexical syntax, as in tag and comment syntax, is not strictly enforced. Browsers became very forgiving, and in HTML 5, recovery from a long list of common errors, including comments with bad syntax, was formalized in the spec. There are pages of error recovery gimmicks defined in the spec, and HTML 5 parsers have to do all that stuff. (This is one reason why html5parser for Python is so slow.) There used to be tricks for doing different things in IE and Firefox based on how they handled errors. On the first error, a browser should display an error bar in the middle of the page, then proceed on a best-effort basis, ignoring most styling. Bad pages would still be readable in emergencies, but they'd be annoying enough to get fixed. (Until you write a web crawler, you don't realize how bad HTML is in the wild. There are low-level syntax errors. There are pages with more than one <html>, <head>, or <body> section, or where those sections are out of order. There are nesting errors which result in page trees a thousand deep. There are tags that don't belong in HTML. There's syntactically incorrect ad code from major vendors.)
- whipoodle 9y agoDefinitely. There was an effort to remedy this with draconian error handling (XHTML), and it badly flopped. Edit: I don't mean "draconian" in a pejorative sense, it is just typically the word used to describe this.
- robertlagrant 9y agoFunny how people think that throwing a syntax error is "Draconian" just because it's HTML.
- jacquesm 9y agoBecause an end user has zero recourse after getting such a syntax error. Those are only useful for developers. Graceful error handling is one of the things that set good applications apart from (really) bad ones.
- 9y ago
- Bartweiss 9y agoInterestingly, I seem to have the opposite complaint about XML parsers. People generate crappy, broken XML all the time, but many of the services that digest it are horribly picky and unwilling to take even valid inputs. Some refuse empty fields and demand they not be included as tags. Some demand a specific list of tags, and insist all be included even if empty. Some clearly have non-recursive parsing, only honor certain depths of tree. Some have completely hard-coded parsers, and only accept certain tags in certain orders. The list goes on. Is this just Sturgeon's Law, where things are broken in both directions? Or is there some predictable reason that some tools become overly permissive, and others overly restrictive?
- captainmuon 9y agoI have the same with JSON. Sure, when I'm using JSON/REST APIs everything works fine, but I deal a lot with JSON on disk, and it happens a lot that you have sloppy JSON because people hand-edit it. Trailing commas and comments are the most common, but sometimes you just have js/python object literals dumped into a file. In every language I work in, I find myself writing a lenient JSON parser. So far, Python, C++, and Nim (for JS, there is already JSON5). I am also working on a whitespace and comment preserving parser that round-trips, so you can change a value in a configuration file without loosing comments. (Currently, I always produce standard JSON so compatibility is not an issue.) I have been thinking about publishing these parsers, but after seeing how much abuse and nasty comments the JSON5 guy got, I have been holding off...
- Zaak 9y agoI would love to see a space and comment preserving json parser. Being able to programmatically update a config file without losing formatting and such sounds like a big win.
- nly 9y ago> I am also working on a whitespace and comment preserving parser that round-trips, so you can change a value in a configuration file without loosing comments. I am 80% done writing such a parser for a over-engineered legacy config file format. It isn't that hard really, you just need a parser framework/library that can output a complete parse tree (a tree where a preorder traversal covers every byte in the input contiguously, exactly once). The remaining 20% of the work is keeping track of the line and column number (almost there), cleaning up the code, and being able to reserialize the AST to a binary format (for reloading in another process where I don't want to have to rerun the original PEG)
- RcouF1uZ4gsC 9y agoI think the benefits of Postel's Maxim, have far outweighed the negatives. One of the reasons why the web took off, is that normal everyday non-geeky people could in a few minutes crank out a webpage that could be viewed by the whole world. The browsers just made it work and generally look OK. If we did not have Postel's Maxim, I think the Internet would have just remained the domain of geeks and not have become what it is today.
- wott 9y agoAgreed, but: > If we did not have Postel's Maxim, I think the Internet would have just remained the domain of geeks and not have become what it is today. That would have been a huge benefit as well.
- zAy0LfpBZLC8mAC 9y agoThat's bullshit. It's not hard to crank out a standards-compliant web page if your browser tells you where what you wrote is broken. Certainly a lot easier than cranking out a web page that looks good in a bunch of browsers that all have their own idea of what HTML means. If you think that browsers just made things work, you never tried to build a web page that actually displayed correctly in both IE and Netscape at the time.
- jasode 9y agoThe IETF title makes it sound like developers everywhere were using Postel's words as guidance. Instead, relaxed or permissive parsing of files/protocols/API even though they have well-documented specifications is an unavoidable emergent phenomenon. The various examples across domains is fascinating: - HTML (missing tags, unpaired tags, etc). Developers may prefer to strictly parse HTML and any non-compliance results in a "page error". But the websurfers want that page data so developers end up guessing the HTML authors intentions and renders the page. - PDF (malformed pdf files that Adobe can read). Lots of utilities out there create bad pdf files and Adobe Acrobat has lots of workarounds to read them. Even though Adobe Inc controls the pdf specification, even they succumb to the pressure of adding workarounds to their parser to render broken pdf files. To add to the insanity, all 3rd-party industrial-strength pdf parsers end up copying Adobe Acrobat's behavior to parse broken pdf files! - Win32 API. Programmers out in the wild will notice an undocumented behavior of an API that's not in the contract and start to depend on it. When a new version of Windows is released that removes the undocumented behavior and breaks the 3rd-party app, the customer blames the "Microsoft Windows upgrade" and not Quicken or videogame company. Raymond Chen has written several articles on Microsoft adding "shims" to Windows codebase to help "guarantee compatibility" for misbehaving apps incorrectly using the Win32 API. If the 3rd-party app is important enough to consumers that it prevents them from upgrading Windows (in other words -- pay Microsoft money), it means Microsoft will bend over backwards to accommodate the badly-written code from the vendor. The external forces are too great to enforce perfect discipline of strict parsing across all parties. Even the big proprietary companies like Microsoft and Adobe can't enforce their own standards-compliant parsing. An open standard from IETF would have no chance at all. We'd like to think of a "computing standard" as some inviolable contract but history has shown it is actually an organic (and unspoken) "social" agreement. This is unavoidable unless we all agree to have all software "approved" by a central authority before anyone can download or use it.
- dozzie 9y ago> Developers may prefer to strictly parse HTML and any non-compliance results in a "page error". But the websurfers want that page data [...] Note that similar thing was happening with SSL/TLS connections for a long time: developers may prefer to strictly verify X.509 certificates, but websurfers want that page data. The result of developers bowing to accepting invalid certificates was hurting everybody. Nowadays the situation is much better, because browser developers made visiting sites with self-signed, expired, or incorrectly named certificates significantly harder.
- amelius 9y agoIf developers can't even be bothered to get protocols right, how the hell are they going to get security right?
- jimrandomh 9y agoThe problem is that there are two very different contexts: users and developers. If a developer is testing their new program by having it interface with your program, you want to be strict in what you accept; this enables them to make their program strict in what it emits, and helps bugs surface early. But if an end-user is hooking your program up to some other program, then you want to be generous in what you'll accept, so that it'll work. Compilers solve this by having an intermediate levels of strictness: warnings, which a developer who's providing input to the compiler is expected to fix, but which an end-user doesn't have to. This captures the benefits of both strict and lax input checking. It helps to make similar distinctions in other contexts, whether that's having an explicit notion of "warnings", or just putting messages in log-file output. It also helps if a format has a well-known extra-strict checker around, which developers can test their programs' output on.
- zAy0LfpBZLC8mAC 9y ago> But if an end-user is hooking your program up to some other program, then you want to be generous in what you'll accept, so that it'll work. No, you absolutely don't. The only reason why you would possibly need that in the first place is because software tends to be far too forgiving. Where software enforces procotol/format compliance, a normal end user essentially does not ever get to see any deviating input because generating software is written to the standard right from the start, and the few cases where that fails, the actual bug gets fixed. Also, you actually can't. When the standard does not specify the meaning of some input, then there is no meaning. Whatever meaning you invent for it is just that: Your invention. The next software likely will interpret things differently. You do not actually have a standardized format when multiple implementations speak the same syntax, they also need to have the same semantics. Just doing something in response to broken input does not produce interoperability, but just undefined behaviour.
- fizixer 9y agoI like the webpage format. They even made sure (apparently) that printing preserves the layout and page numbering.
- 0xffff2 9y agoInteresting. I find it to be one of the most egregious examples of bad skeuomorphic design. It makes no sense to me to break documents up into pages when viewing them on the web, and in Firefox's print preview it looks like the "pages" won't even print properly (the last line or two gets bumped to the next page).
- zAy0LfpBZLC8mAC 9y agoExcept that's not really what it is. RFCs predate the web, and were originally written in this format, and with existing standards you don't want to unnecessarily change anything, because you never know whether you maybe change the meaning somewhere as a side effect, and you also want to have consistency between standards from the same standards body, so the format was kept, and also, a plain text format was long seen as having the big advantage that it is not easily corrupted by bad rendering software, so there was just some HTML markup added after the fact to help with navigation and such, much while otherwise preserving the presentation. However, the IETF is moving to markup as the authoritative source, so the pseudo plain text version might be losing relevance.
- 0xffff2 9y agoI see what you're saying, but I disagree that it's a substantial rebuttal to my comment. For any IETF document that wasn't originally drafted on a typewriter, this style is absolutely skeumorphic. I don't necessarily mind it as a default, but I really wish the pure plain-text document ommited the headers and footers.
- zAy0LfpBZLC8mAC 9y agoNah, it's not really skeuomorphic because it doesn't really imitate, at least until recently. The authoritative format was plain text, and that was intended to be printable (originally probably on what were in effect computer-driven typewriters), and if you want to have printable plain text, you kindof have to have headers and footers on every "page" (and page break characters between pages in order to accomodate different paper sizes, which are actually there if you look closely at the plain text version). The HTML version of those RFCs does not so much imitate the format as it is a rendering that is supposed to reproduce the authoritative document as true to the original as possible, so as to not risk any unintended changes in meaning, while adding some of the benefits of having hypertext.
- akkartik 9y agohttps://www.joelonsoftware.com/2008/03/17/martian-headsets https://www.joelonsoftware.com/2008/03/17/martian-headsets
- js2 9y agoWhy do the text versions of this draft have the header "Elephants Out, Donkeys In?" Is that political commentary unrelated to the draft?