5 ms·
The information can be useful. Some years back I hit on a notion of how to rate websites, domains, and even TLDs based on the prevalence of specific terms with
by dredmorbius 4y ago
The information can be useful.
Some years back I hit on a notion of how to rate websites, domains, and even TLDs based on the prevalence of specific terms within them, as returned by Google search.
I came up with a list of 100 search terms in the Foreign Policy Top 100 Global Thinkers list. I added a few searches that should at least proxy for English-language texts using frequently-used but not stopwords (the word "this" was one of those). And arbitrarily chose the string "Kim Kardashian" to represent non-salient content.
That gave me a touch over 100 terms, and I identified roughly 100 target sites (sites, domains, TLDs).
The method was to run a Google web search on each of these and scrape the reported number of hits. That meant something north of 10,000 Google web searches, which I automated with a creative application of delays (up to several minutes), running over a week or two. Anti-bot tactics deployed by Google make this all but impossible now, though there's a service which (for a price) offers a directly-queriable web index so far as I'm aware, which might make for some interesting further research.
I summarized and posted the results here: <https://old.reddit.com/r/dredmorbius/comments/3hp41w/tracking_the_conversation_fp_global_100_thinkers/ https://old.reddit.com/r/dredmorbius/comments/3hp41w/trackin...>
But contrary to your assertion, estimates of total results even if not directly viewable through Google SERPs are* of potential value.