5 ms·

As of HTML5, web standards specify how to render invalid HTML as well, precisely to avoid this problem: https://html.spec.whatwg.org/multipage/parsing.html#par

by Zarel 6y ago

As of HTML5, web standards specify how to render invalid HTML as well, precisely to avoid this problem: https://html.spec.whatwg.org/multipage/parsing.html#parse-errors https://html.spec.whatwg.org/multipage/parsing.html#parse-er... In addition, most of the suggestions in the link (leaving off <html> and <head>, leaving off quotes for attributes, not closing <p>) are valid HTML in the first place.

lhorie 6y ago
To add to this, one of suggestions in the article was to drop the doctype. While that puts the browser in tag soup parsing mode rather than html5, in practice browsers don't actually start diverging much in behavior until you get to truly messed up markup, like malformed tags. So even the more egregious suggestions here are probably still fine, provided that you have equally egregious CSS hacks to get around ancient box model quirks
oefrha 6y ago
True, and the suggestions in the article are pretty tame. But let’s set aside the article and talk about invalid HTML for a minute. I write scrapers a lot (not the irresponsible kind and never for monetary gains) and invalid HTML, while technically parseable and to spec, are often a pain in the ass. You have to bring in a full blown HTML5 parser, and they could be way slower (e.g. lxml.html vs html5lib). Depending on your language of choice there might not even be a to-spec HTML5 parser available. So, just close your damn tags (except self-closing ones), and close them in order, it’s not hard, the size increase is minimal, it will help with your own sanity and people will thank you for it.
- thinkloop 6y ago
  For many sites this is an endorsement not to close tags.
  - oefrha 6y ago
    It’s an annoyance, not a meaningful hindrance.
- goto11 6y ago
  There are a lot more web sites than scrapers though. Why push the work to all the web sites in the world instead of fixing your scraper? But in any case, you are never getting all the web sites in the world to validate, so what does it matter if you can get some of them to validate? You still have to handle the rest.
  - oefrha 6y ago
    "it will help with your own sanity" Writing unclosed/out-of-order tag soup is extra work, not less work. What a weird hill to die on.
    - goto11 6y ago
      What people don't get about the web is that there is a significant (and growing) number of pages written by people who are now dead. So all the well-intentioned ideas about "cleaning up" html - it's not going to happen. It's not a question about being pro or con closing tags or which is "best". The question is about the billions of pages which already exist and is not going to change. If you write a web scraper, that is just the reality you work with.
      - oefrha 6y ago
        Where did I call for people to clean up existing HTML? Nowhere. Did I ask people to not follow bad advice (intentionally not closing non-self-closing tags just for the hell of it)? Absolutely.
- deleted 6y ago
  [deleted]
- chrismorgan 6y ago
  I extremely strongly disagree with your points and conclusion here: why should I, a web author, care because you’re using a slow language that doesn’t have a real HTML parser available? You want to parse my HTML, use an HTML parser. Parsing an HTML document without a correct HTML parser is just wrong. (I decline to call it an HTML 5 parser, because it’s not strictly that any more, and this stuff is from more than ten years ago: get with the program!) I’d lump it in with using regular expressions to parse HTML: useful in certain situations, but not wise for the general case. So you have a fast but wrong HTML parser? Find a correct HTML parser. If lxml.html doesn’t parse correctly (I don’t know whether it does or not, I haven’t checked—but suppose it doesn’t), then I maintain that you should never under any circumstances use it in new code. It’s an artefact of twelve years ago, and it’s bad. So you have a correct but slow HTML parser, and you’re not happy with that? I could say “if you care about performance why are you using Python anyway”, but that would be disingenuous—though some of those sentiments can still reasonably apply. Anyway, your remedy is to find a fast and correct one. I confess myself a tad surprised to find no stable or featureful bindings to html5ever, which is one of the best things along these lines. https://github.com/SimonSapin/html5ever-python https://github.com/SimonSapin/html5ever-python and https://pypi.org/project/htmlpyever/ https://pypi.org/project/htmlpyever/ are two things that at least start on this, but it looks like nothing interesting’s happened in this space for about three years. Huh. But anyway: I decline to adjust my authoring practices because you refuse to use the right tools. Enough other people use the right tools that I don’t need to care. It’s rather like the XHTML/HTML situation if you squint: the only reason to use XHTML, which would reject an invalid document (while HTML would let your nominally invalid document do something useful), would be if something you were interacting with required it.
  - oefrha 6y ago
    If your authoring practice is producing wrongly closed tag soup that happens to work (making sure it happens to work takes longer than writing it correctly in the first place), making it unnecessarily harder for everyone including you and your coworkers, then you simply suck as a developer. Of course I'm not paying you, so whatever.
    - 6y ago
Izkata 6y ago
> As of HTML5, web standards specify how to render invalid HTML as well Ah, but this post is (seemingly accidentally) recommending you use quirks mode instead of HTML5: > * <!DOCTYPE html> is required by the HTML spec (but not the XHTML spec). Most of browsers will not care if you omit it. It was created for backwards compatibility with prior HTML versions. You can omit it in minimal documents.

by Zarel 6y ago

lhorie 6y ago

To add to this, one of suggestions in the article was to drop the doctype. While that puts the browser in tag soup parsing mode rather than html5, in practice browsers don't actually start diverging much in behavior until you get to truly messed up markup, like malformed tags. So even the more egregious suggestions here are probably still fine, provided that you have equally egregious CSS hacks to get around ancient box model quirks

oefrha 6y ago

True, and the suggestions in the article are pretty tame. But let’s set aside the article and talk about invalid HTML for a minute. I write scrapers a lot (not the irresponsible kind and never for monetary gains) and invalid HTML, while technically parseable and to spec, are often a pain in the ass. You have to bring in a full blown HTML5 parser, and they could be way slower (e.g. lxml.html vs html5lib). Depending on your language of choice there might not even be a to-spec HTML5 parser available. So, just close your damn tags (except self-closing ones), and close them in order, it’s not hard, the size increase is minimal, it will help with your own sanity and people will thank you for it.

thinkloop 6y ago
For many sites this is an endorsement not to close tags.
- oefrha 6y ago
  It’s an annoyance, not a meaningful hindrance.
goto11 6y ago
There are a lot more web sites than scrapers though. Why push the work to all the web sites in the world instead of fixing your scraper? But in any case, you are never getting all the web sites in the world to validate, so what does it matter if you can get some of them to validate? You still have to handle the rest.
- oefrha 6y ago
  "it will help with your own sanity" Writing unclosed/out-of-order tag soup is extra work, not less work. What a weird hill to die on.
  - goto11 6y ago
    What people don't get about the web is that there is a significant (and growing) number of pages written by people who are now dead. So all the well-intentioned ideas about "cleaning up" html - it's not going to happen. It's not a question about being pro or con closing tags or which is "best". The question is about the billions of pages which already exist and is not going to change. If you write a web scraper, that is just the reality you work with.
    - oefrha 6y ago
      Where did I call for people to clean up existing HTML? Nowhere. Did I ask people to not follow bad advice (intentionally not closing non-self-closing tags just for the hell of it)? Absolutely.
deleted 6y ago
[deleted]
chrismorgan 6y ago
I extremely strongly disagree with your points and conclusion here: why should I, a web author, care because you’re using a slow language that doesn’t have a real HTML parser available? You want to parse my HTML, use an HTML parser. Parsing an HTML document without a correct HTML parser is just wrong. (I decline to call it an HTML 5 parser, because it’s not strictly that any more, and this stuff is from more than ten years ago: get with the program!) I’d lump it in with using regular expressions to parse HTML: useful in certain situations, but not wise for the general case. So you have a fast but wrong HTML parser? Find a correct HTML parser. If lxml.html doesn’t parse correctly (I don’t know whether it does or not, I haven’t checked—but suppose it doesn’t), then I maintain that you should never under any circumstances use it in new code. It’s an artefact of twelve years ago, and it’s bad. So you have a correct but slow HTML parser, and you’re not happy with that? I could say “if you care about performance why are you using Python anyway”, but that would be disingenuous—though some of those sentiments can still reasonably apply. Anyway, your remedy is to find a fast and correct one. I confess myself a tad surprised to find no stable or featureful bindings to html5ever, which is one of the best things along these lines. https://github.com/SimonSapin/html5ever-python https://github.com/SimonSapin/html5ever-python and https://pypi.org/project/htmlpyever/ https://pypi.org/project/htmlpyever/ are two things that at least start on this, but it looks like nothing interesting’s happened in this space for about three years. Huh. But anyway: I decline to adjust my authoring practices because you refuse to use the right tools. Enough other people use the right tools that I don’t need to care. It’s rather like the XHTML/HTML situation if you squint: the only reason to use XHTML, which would reject an invalid document (while HTML would let your nominally invalid document do something useful), would be if something you were interacting with required it.
- oefrha 6y ago
  If your authoring practice is producing wrongly closed tag soup that happens to work (making sure it happens to work takes longer than writing it correctly in the first place), making it unnecessarily harder for everyone including you and your coworkers, then you simply suck as a developer. Of course I'm not paying you, so whatever.
  - 6y ago

Izkata 6y ago

> As of HTML5, web standards specify how to render invalid HTML as well Ah, but this post is (seemingly accidentally) recommending you use quirks mode instead of HTML5: > * <!DOCTYPE html> is required by the HTML spec (but not the XHTML spec). Most of browsers will not care if you omit it. It was created for backwards compatibility with prior HTML versions. You can omit it in minimal documents.