Ha. I've had the same thoughts many times. Never thought anyone else felt this way. At some stage, people started to get weird ideas about URL schemes (aka "

Ha. I've had the same thoughts many times. Never thought anyone else felt this way. At some stage, people started to get weird ideas about URL schemes (aka "API's") like it's some sort of marketing thing. API keys. WTF? Imagine if early Google was required to get an "API key" for every site they crawled. Who came up with this silly idea of "API keys"? In the 90's, I remember decipering URL schemes in order to do crawling long before anyone used the term "API's" to describe URL's. They are usually quite predictable, since almost every website used the same software. And they're even more predictable today. Crwaling got easier. Hmmm, I wonder why. I also remember how most URL's used to be non-descriptive. Not true anyomre. Google has changed everything. Sites _want_ to be crawled now. But funnily enough they think they can make money from it. Heh, good luck with that. If you want to charge money, then put the data behind a password and a paywall. And watch traffic plummet.