4 ms·
This is total bullshit. This is assuming that you are parsing well formatted, modern HTML. Anyone who has had to data mine HTML on a regular basis knows regula
by PHP-TROLL 17y ago
This is total bullshit. This is assuming that you are parsing well formatted, modern HTML.
Anyone who has had to data mine HTML on a regular basis knows regular expressions are the only way to go because most web pages are clusterfucks of invalid, font-tag laden, id-less, ancient piles of shit.
Any XML parser I threw at the sites I was mining ran away crying with it's pants soaked with urine. So climb down your ivory tower, because XML parsers may be the "correct" way to get data out of HTML, but they are near worthless for most situations.
- psadauskas 17y agoThats because you're using an XML parser to parse HTML. You should use an HTML parser, like hpricot or beautifulsoup.
- PHP-TROLL 17y agoYeah when I said XML parsers I was speaking generically. I tried many HTML and XML parsers as well has HTML sanitizers. They worked, maybe, 60% of the time. Hpricot regularly segfaulted, beautifulsoup didn't even begin to work. Even if I ran the HTML through tidy first it would fail to parse. Most people are ignorant of how bad most of the HTML on the web is, it is simply unparseable.
- coderdude 17y ago>>Most people are ignorant of how bad most of the HTML on the web is, it is simply unparseable. And somehow using a regex to guess at all the possible combinations of "badness" is the correct way to go? You seem lost my friend. Everything you're saying is backwards to what someone who actually parses a lot of HTML "in the wild" would say on the topic. It sounds more like you're trying to cuss your way into being correct.
- PHP-TROLL 17y agoI'm not talking about parsing HTML. I'm talking about getting data out of HTML which is completely different. One doesn't need to guess at all possible combinations of badness using a regex, you just need to find the data you need and adjust the regex to the badness of that particular page. Do you know anyone that mines for data out of HTML "in the wild"? I didn't think so.
- jm4 17y agoMost people are ignorant of how bad most of the HTML on the web is, it is simply unparseable. If that was the case the web would be unusable. Any decent web browser will parse almost any garbage you throw at it. It can get complicated because you find yourself guessing at what the author's intent might have been or moving things around to make it semi-sane, but it's not only possible; it's done every day.
- spuz 17y agoIt's not total bullshit. Did you get to the end of the article before commenting on it? I think Jeff hit the nail on the head; sure regex is generally not very good a parsing HTML, but in certain situations it is the right tool for the job. You seem to have considered your options and chosen the right tool. The point of the article is that often you don't need to teach someone which tool they should be using, but how to recognise that there might be better solutions to the problem they're solving and how to go about finding that solution.