Your timeline is completely confused. ASPX came out in 2002, after HTML4. HTML2 was the era when CGI scripts were dominant. Next, HTML2+SGML were also well-des

Your timeline is completely confused. ASPX came out in 2002, after HTML4. HTML2 was the era when CGI scripts were dominant. Next, HTML2+SGML were also well-designed, and people weren't just doing regexp. The mess didn't come in until HTML3 and even more so, HTML4. Today, it's easy to parse /specific/ pages. If I want to automate one web page, and it's well-formed, HTML5+AJAX makes that easy. However, in contrast to HTML2, it's very hard to parse pages /generically/. That's why I gave the example of Altavista and a11y tools, which need to work with any web site. Try to make something like that today: a generic search spider, a web browser, or an a11y tool. See how far you get talking to JSON endpoints. They're often easy enough to reverse-engineer for a specific web site, but you need a human-in-the-loop for each web site. With HTML2, one would build tools which could work with /any/ web site. And boy were there a lot of tools. Look at all the web browsers of the nineties, and the innovation there.