3 ms·
For years and years I ran a web service that scraped another site's HTML to extract data. There were other APIs doing the same thing. They used a proper HTML pa
by electroly 1y ago
For years and years I ran a web service that scraped another site's HTML to extract data. There were other APIs doing the same thing. They used a proper HTML parser, and I just used the moral equivalent of String.IndexOf() to walk a cursor through the text to locate the start and end of strings I wanted and String.Substring() to extract them. Theirs were slow and sometimes broke when unrelated structural HTML changes were made. Mine was a straight linear scan over the text and didn't care at all about the HTML in between the parts I was scraping. It was even an arbitrarily recursive data structure I was parsing, too. I was able to tell at each step, by counting the end and start tags, how many levels up or down I had moved without building any tree structures in memory. Worked great, reliably, and I'd do it again.